Systems and methods for image processing

ABSTRACT

The present disclosure provides systems and methods for image processing. The method may include obtaining an initial image; obtaining an intermediate image corresponding to the initial image, the intermediate image including pixels or voxels associated with at least a portion of a target object in the initial image; obtaining a trained processing model; and generating, based on the initial image and the intermediate image, a target image associated with the target object using the trained processing model.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority of Chinese Patent Application No.201811627428.3, filed on Dec. 28, 2018, the contents of which are herebyincorporated by reference.

TECHNICAL FIELD

The disclosure generally relates to image processing, and inparticularly, to systems and methods for segmenting an image.

BACKGROUND

With the development of imaging technology, angiographic imaging (e.g.,digital subtraction angiography (DSA)) has been widely used in medicaldiagnosis. Accordingly, blood vessel segmentation (i.e., extraction ofblood vessel images) techniques can be used to segment (or extract)blood vessel(s) from images generated by angiographic imaging, therebyfacilitating clinical diagnosis of vascular diseases. For example, alocation, size, and/or shape of a lesion in a coronary artery may beidentified or determined based on a coronary artery image. However, insome situations, such as for a coronary artery image, it is difficult tosegment the coronary artery due to a relatively low contrast of thecoronary artery and/or uneven distribution of contrast agents.Therefore, it is desirable to provide systems and methods for segmentingblood vessels accurately.

SUMMARY

In one aspect of the present disclosure, a method for image processingis provided. The method may include: obtaining an initial image;obtaining an intermediate image corresponding to the initial image, theintermediate image including pixels or voxels associated with at least aportion of a target object in the initial image; obtaining a trainedprocessing model; and generating, based on the initial image and theintermediate image, a target image associated with the target objectusing the trained processing model.

In some embodiments, the obtaining an intermediate image may include:generating the intermediate image by processing the initial image usingat least one filter.

In some embodiments, the generating the intermediate image by processingthe initial image using at least one filter may include: determining aHessian matrix corresponding to each pixel or voxel of the initialimage; determining, based on the Hessian matrix, at least onecharacteristic value corresponding to the each pixel or voxel;determining a response value corresponding to the each pixel or voxel byenhancing, using a first filter of the at least one filter, the at leastone characteristic value corresponding to the each pixel or voxel; andgenerating, based on a plurality of response values corresponding to aplurality of pixels or voxels of the initial image, the intermediateimage.

In some embodiments, the first filter includes at least one of aGaussian filter, a tubular filter, a linear filter, or a Wiener filter.

In some embodiments, the generating the intermediate image by processingthe initial image using at least one filter may further include:smoothing the initial image using a second filter of the at least onefilter.

In some embodiments, the second filter may include at least one of aGaussian filter, a linear filter, or a Wiener filter.

In some embodiments, the generating the intermediate image by processingthe initial image using at least one filter may include: generating oneor more smoothed initial images by smoothing the initial image using oneor more second filters of the at least one filter; determining a Hessianmatrix corresponding to each pixel or voxel of each smoothed initialimage of the one or more smoothed initial images; determining, based onthe Hessian matrix, at least one characteristic value corresponding tothe each pixel or voxel of the each smoothed initial image; determininga response value corresponding to the each pixel or voxel of the eachsmoothed initial image by enhancing, using a first filter of the atleast one filter, the at least one characteristic value corresponding tothe each pixel or voxel of the each smoothed initial image; determining,based on one or more response values corresponding to pixels or voxelsin the one or more smoothed initial images, a target response valuecorresponding to each pixel or voxel of the initial image; andgenerating, based on a plurality of target response values correspondingto a plurality of pixels or voxels of the initial image, theintermediate image.

In some embodiments, the generating a target image may include: fusingthe initial image and the intermediate image to obtain a fusion image;and inputting the fusion image into the trained processing model togenerate the target image.

In some embodiments, the fusing the initial image and the intermediateimage may include: generating the fusion image by processing a firstvalue of each pixel or voxel of the initial image with a second value ofa corresponding pixel or voxel of the intermediate image.

In some embodiments, the processing a first value of each pixel or voxelof the initial image with a second value of a corresponding pixel orvoxel of the intermediate image may include: determining a value of apixel or voxel of the fusion image based on a sum or product of thefirst value and the second value.

In some embodiments, the generating a target image may include:inputting, in a parallel mode, the initial image and the intermediateimage into two different input channels of the trained processing modelto generate the target image.

In some embodiments, the method may further include: obtaining aninitial processing model; and training the initial processing model toobtain the trained processing model.

In some embodiments, the trained processing model may be generatedaccording to a process, the process may include: obtaining an initialprocessing model; obtaining a plurality of training samples, theplurality of training samples including a plurality of initial sampleimages and a plurality of intermediate sample images corresponding tothe plurality of initial sample images; and generating the trainedprocessing model by training the initial processing model using theplurality of training samples.

In some embodiments, the trained processing model may be configured tosegment, based on the intermediate image, the target object from theinitial image.

In some embodiments, the trained processing model may include a coarsesegmentation network and a fine segmentation network.

In some embodiments, the trained processing model may be a trained V-Netneural network model.

In some embodiments, the method may further include: updating the targetimage, including: extracting a largest connected domain in the targetimage as an updated target image.

In some embodiments, the target object may include a blood vessel.

In another aspect of the present disclosure, a system for imageprocessing is provided. The system may include: at least one storagedevice storing executable instructions, and at least one processor incommunication with the at least one storage device. When executing theexecutable instructions, the at least one processor may cause the systemto perform operations including: obtaining an initial image; obtainingan intermediate image corresponding to the initial image, theintermediate image including pixels or voxels associated with at least aportion of a target object in the initial image; obtaining a trainedprocessing model; and generating, based on the initial image and theintermediate image, a target image associated with the target objectusing the trained processing model.

In another aspect of the present disclosure, a non-transitory computerreadable medium is provided. The non-transitory computer readable mediummay include at least one set of instructions for image processing,wherein when executed by one or more processors of a computing device,the at least one set of instructions may cause the computing device toperform a method. The method may include: obtaining an initial image;obtaining an intermediate image corresponding to the initial image, theintermediate image including pixels or voxels associated with at least aportion of a target object in the initial image; obtaining a trainedprocessing model; and generating, based on the initial image and theintermediate image, a target image associated with the target objectusing the trained processing model.

Additional features will be set forth in part in the description whichfollows, and in part will become apparent to those skilled in the artupon examination of the following and the accompanying drawings or maybe learned by production or operation of the examples. The features ofthe present disclosure may be realized and attained by practice or useof various aspects of the methodologies, instrumentalities, andcombinations set forth in the detailed examples discussed below.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is further described in terms of exemplaryembodiments. These exemplary embodiments are described in detail withreference to the drawings. These embodiments are non-limiting exemplaryembodiments, in which like reference numerals represent similarstructures throughout the several views of the drawings, and wherein:

FIG. 1 is a schematic diagram illustrating an exemplary applicationscenario of an imaging processing system according to some embodimentsof the present disclosure;

FIG. 2 is a schematic diagram illustrating exemplary hardware and/orsoftware components of an exemplary computing device according to someembodiments of the present disclosure;

FIG. 3 is a schematic diagram illustrating exemplary hardware and/orsoftware components of an exemplary mobile device according to someembodiments of the present disclosure;

FIG. 4A is a block diagram illustrating an exemplary processing deviceaccording to some embodiments of the present disclosure;

FIG. 4B is a block diagram illustrating another exemplary processingdevice according to some embodiments of the present disclosure;

FIG. 5 is a flowchart illustrating an exemplary process for generating atarget image associated with one or more target objects according tosome embodiments of the present disclosure;

FIG. 6 is a flowchart illustrating an exemplary process for generating atarget image associated with one or more target objects according tosome embodiments of the present disclosure;

FIG. 7A is a flowchart illustrating an exemplary process for determininga target image associated with one or more target objects according tosome embodiments of the present disclosure;

FIG. 7B is a flowchart illustrating an exemplary process for training aprocessing model according to some embodiments of the presentdisclosure;

FIG. 8 is a flowchart illustrating an exemplary process for determininga target image associated with one or more target objects according tosome embodiments of the present disclosure;

FIG. 9 is a diagram illustrating an exemplary process for generating anintermediate image and a target image according to some embodiments ofthe present disclosure;

FIG. 10 is a block diagram illustrating an exemplary medical deviceaccording to some embodiments of the present disclosure;

FIG. 11A shows an exemplary initial image according to some embodimentsof the present disclosure;

FIG. 11B shows an exemplary intermediate image corresponding to theinitial image according to some embodiments of the present disclosure;

FIG. 12A shows an exemplary image generated based on a single sourcemodel, an exemplary image generated based on a multi-source model, andan exemplary gold standard image according to some embodiments of thepresent disclosure; and

FIG. 12B shows an exemplary image generated without extracting a largestconnected domain, an exemplary image generated with extracting thelargest connected domain, and an exemplary gold standard image accordingto some embodiments of the present disclosure.

DETAILED DESCRIPTION

In the following detailed description, numerous specific details are setforth by way of examples in order to provide a thorough understanding ofthe relevant disclosure. However, it should be apparent to those skilledin the art that the present disclosure may be practiced without suchdetails. In other instances, well-known methods, procedures, systems,components, and/or circuitry have been described at a relativelyhigh-level, without detail, in order to avoid unnecessarily obscuringaspects of the present disclosure. Various modifications to thedisclosed embodiments will be readily apparent to those skilled in theart, and the general principles defined herein may be applied to otherembodiments and applications without departing from the spirit and scopeof the present disclosure. Thus, the present disclosure is not limitedto the embodiments shown, but to be accorded the widest scope consistentwith the claims.

The terminology used herein is for the purpose of describing particularexample embodiments only and is not intended to be limiting. As usedherein, the singular forms “a,” “an,” and “the” may be intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprise,”“comprises,” and/or “comprising,” “include,” “includes,” and/or“including,” when used in this specification, specify the presence ofstated features, integers, steps, operations, elements, and/orcomponents, but do not preclude the presence or addition of one or moreother features, integers, steps, operations, elements, components,and/or groups thereof. It will be understood that the term “object” and“subject” may be used interchangeably as a reference to a thing thatundergoes a treatment and/or an imaging procedure in a radiation systemof the present disclosure.

It will be understood that the term “system,” “engine,” “unit,”“module,” and/or “block” used herein are one method to distinguishdifferent components, elements, parts, section or assembly of differentlevel in ascending order. However, the terms may be displaced by anotherexpression if they achieve the same purpose.

Generally, the word “module,” “unit,” or “block,” as used herein, refersto logic embodied in hardware or firmware, or to a collection ofsoftware instructions. A module, a unit, or a block described herein maybe implemented as software and/or hardware and may be stored in any typeof non-transitory computer-readable medium or another storage device. Insome embodiments, a software module/unit/block may be compiled andlinked into an executable program. It will be appreciated that softwaremodules can be callable from other modules/units/blocks or themselves,and/or may be invoked in response to detected events or interrupts.Software modules/units/blocks configured for execution on computingdevices (e.g., processor 210 as illustrated in FIG. 2) may be providedon a computer-readable medium, such as a compact disc, a digital videodisc, a flash drive, a magnetic disc, or any other tangible medium, oras a digital download (and can be originally stored in a compressed orinstallable format that needs installation, decompression, or decryptionprior to execution). Such software code may be stored, partially orfully, on a storage device of the executing computing device, forexecution by the computing device. Software instructions may be embeddedin firmware, such as an EPROM. It will be further appreciated thathardware modules/units/blocks may be included in connected logiccomponents, such as gates and flip-flops, and/or can be included ofprogrammable units, such as programmable gate arrays or processors. Themodules/units/blocks or computing device functionality described hereinmay be implemented as software modules/units/blocks but may berepresented in hardware or firmware. In general, themodules/units/blocks described herein refer to logicalmodules/units/blocks that may be combined with othermodules/units/blocks or divided into sub-modules/sub-units/sub-blocksdespite their physical organization or storage. The description mayapply to a system, an engine, or a portion thereof.

It will be understood that when a unit, engine, module or block isreferred to as being “on,” “connected to,” or “coupled to,” anotherunit, engine, module, or block, it may be directly on, connected orcoupled to, or communicate with the other unit, engine, module, orblock, or an intervening unit, engine, module, or block may be present,unless the context clearly indicates otherwise. As used herein, the term“and/or” includes any and all combinations of one or more of theassociated listed items.

As used herein, a representation of an object (e.g., a patient, or aportion thereof) in an image may be referred to the object for brevity.For instance, a representation of an organ or tissue (e.g., the heart,the liver, a lung, a blood vessel, etc., of a patient) in an image maybe referred to as the organ or tissue for brevity. As used herein, anoperation on a representation of an object in an image may be referredto as an operation on the object for brevity. For instance, asegmentation of a portion of an image including a representation of anorgan or tissue (e.g., the heart, the liver, a lung, a blood vessel,etc., of a patient) from the image may be referred to as a segmentationof the organ or tissue for brevity.

These and other features, and characteristics of the present disclosure,as well as the methods of operation and functions of the relatedelements of structure and the combination of parts and economies ofmanufacture, may become more apparent upon consideration of thefollowing description with reference to the accompanying drawings, allof which form a part of this disclosure. It is to be expresslyunderstood, however, that the drawings are for the purpose ofillustration and description only and are not intended to limit thescope of the present disclosure. It is understood that the drawings arenot to scale.

The flowcharts used in the present disclosure illustrate operations thatsystems implement according to some embodiments of the presentdisclosure. It is to be expressly understood the operations of theflowcharts may be implemented not in order. Conversely, the operationsmay be implemented in inverted order, or simultaneously. Moreover, oneor more other operations may be added to the flowcharts. One or moreoperations may be removed from the flowcharts.

Provided herein are systems and methods for segmenting a target objectfrom an initial image. A system may include at least one storage devicestoring executable instructions, and at least one processor incommunication with the at least one storage device. When executing theexecutable instructions, the at least one processor may cause the systemto obtain an initial image. The system may obtain an intermediate imagecorresponding to the initial image. The intermediate image may includepixels or voxels associated with at least a portion of a target objectin the initial image. The system may obtain a trained processing model.The system may generate, based on the initial image and the intermediateimage, a target image associated with the target object using thetrained processing model.

Accordingly, the system may input two images, the initial imageincluding a representation of a subject and the intermediate imageincluding a coarse representation of a target object of the subject,into the trained processing model to generate a target image associatedwith the target object. The systems and methods of the presentdisclosure can be conveniently applied in various clinical applications(e.g., blood vessel segmentation, bone segmentation, tracheasegmentation, etc.) including, for example, the segmentation of targetobjects with relatively complex structures (e.g., a coronary artery).Merely by way of example, the initial image may include a representationof the heart, and the intermediate image may include a coarserepresentation of blood vessel(s) (e.g., a coronary artery). The systemsand methods may generate a target image including a fine representationof the blood vessel(s) (e.g., an image including a fine representationof the coronary artery). Accordingly, the systems and methods canimprove the accuracy and efficiency of blood vessel (e.g., coronaryartery) segmentation.

It should be understood that application scenarios of systems andmethods disclosed herein are only some exemplary embodiments providedfor the purposes of illustration, and not intended to limit the scope ofthe present disclosure. For persons having ordinary skills in the art,multiple variations and modifications may be made under the teachings ofthe present disclosure.

FIG. 1 is a schematic diagram illustrating an exemplary applicationscenario of an image processing system according to some embodiments ofthe present disclosure. As shown in FIG. 1, the image processing system100 may include an imaging device 110, a processing device 120, astorage device 130, one or more terminals 140, and a network 150.

The components in the image processing system 100 may be connected inone or more of various ways. Merely by way of example, the imagingdevice 110 may be connected to the processing device 120 through thenetwork 150. As another example, the imaging device 110 may be connectedto the processing device 120 directly as indicated by the bi-directionalarrow in dotted lines linking the imaging device 110 and the processingdevice 120. As a further example, the terminal(s) 140 may be connectedto another component of the image processing system 100 (e.g., theprocessing device 120) via the network 150. As still a further example,the terminal(s) 140 may be connected to the processing device 120directly as illustrated by the dotted arrow in FIG. 1. As still afurther example, the storage device 130 may be connected to anothercomponent of the image processing system 100 (e.g., the processingdevice 120) directly or through the network 150.

The imaging device 110 may be configured to acquire imaging datarelating to at least one part of a subject. The imaging device 110 mayscan the subject or a portion thereof that is located within itsdetection region and generate imaging data relating to the subject orthe portion thereof. The imaging data may include an image (e.g., animage slice), projection data, or a combination thereof. In someembodiments, the imaging data may be two-dimensional (2D) imaging data,three-dimensional (3D) imaging data, four-dimensional (4D) imaging data,or the like, or any combination thereof. The subject may be biologicalor non-biological. For example, the subject may include a patient, aman-made object, etc. As another example, the subject may include aspecific portion, organ, and/or tissue of the patient. As a furtherexample, the subject may include the head, the neck, the thorax, theheart, the stomach, a blood vessel, soft tissue, a tumor, nodules, orthe like, or any combination thereof.

In some embodiments, the imaging device 110 may be implemented in aconfiguration of a single modality imaging device. For example, theimaging device 110 may include a digital subtraction angiography (DSA)device, a positron emission tomography (PET) device, a single-photonemission computed tomography (SPECT) device, a magnetic resonanceimaging (MRI) device (also referred to as an MR device, an MR scanner),a computed tomography (CT) device, an ultrasonography scanner, a digitalradiography (DR) scanner, or the like, or any combination thereof. Insome embodiments, the imaging device 110 may be implemented in aconfiguration of a multi-modality imaging device. Exemplarymulti-modality imaging devices may include a PET-CT device, a PET-MRIdevice, or the like, or a combination thereof.

The processing device 120 may process data and/or information obtainedfrom the imaging device 110, the terminal(s) 140, and/or the storagedevice 130. For example, the processing device 120 may obtain an initialimage. The processing device 120 may obtain an intermediate imagecorresponding to the initial image. The intermediate image may includepixels or voxels associated with at least a portion of a target objectin the initial image. The processing device 120 may obtain a trainedprocessing model. The processing device 120 may generate, based on theinitial image and the intermediate image, a target image associated withthe target object using the trained processing model. The trainedprocessing model may be updated from time to time, e.g., periodically ornot, based on a sample set that is at least partially different from theoriginal sample set from which the trained processing model isoriginally determined. For instance, the trained processing model may beupdated based on a sample set including new samples that are notincluded in the original sample set. In some embodiments, thedetermination and/or updating of the trained processing model may beperformed on a processing device, while the application of the trainedprocessing model may be performed on a different processing device. Insome embodiments, the determination and/or updating of the trainedprocessing model may be performed on a processing device of a systemdifferent from the image processing system 100 (or a server differentfrom a server including the processing device 120 on which theapplication of the trained processing model is performed). For instance,the determination and/or updating of the trained processing model may beperformed on a first system of a vendor who provides and/or maintainssuch a processing model and/or has access to training samples used todetermine and/or update the trained processing model, while imagesegmentation based on the provided trained processing model may beperformed on a second system of a client of the vendor. In someembodiments, the determination and/or updating of the trained processingmodel may be performed online in response to a request for imagesegmentation. In some embodiments, the determination and/or updating ofthe trained processing model may be performed offline.

In some embodiments, the processing device 120 may be a computer, a userconsole, a single server or a server group, etc. The server group may becentralized or distributed. In some embodiments, the processing device120 may be local or remote. For example, the processing device 120 mayaccess information and/or data stored in the imaging device 110, theterminal(s) 140, and/or the storage device 130 via the network 150. Asanother example, the processing device 120 may be directly connected tothe imaging device 110, the terminal(s) 140 and/or the storage device130 to access stored information and/or data. In some embodiments, theprocessing device 120 may be implemented on a cloud platform. Merely byway of example, the cloud platform may include a private cloud, a publiccloud, a hybrid cloud, a community cloud, a distributed cloud, aninter-cloud, a multi-cloud, or the like, or any combination thereof. Insome embodiments, the processing device 120 may be implemented by acomputing device 200 having one or more components as described in FIG.2.

In some embodiments, the processing device 120 may process data and/orinformation obtained from an external resource. For example, theprocessing device 120 may obtain a trained processing model from a thirdparty (e.g., an external storage device of a medical institution, apublic service organization, or a medical company, or the like) thatprovides the trained processing model via the network 150. Theprocessing device 120 may generate a target image using the trainedprocessing model. In some embodiments, the processing device 120, or aportion of the processing device 120 may be integrated into the imagingdevice 110.

The storage device 130 may store data, instructions, and/or any otherinformation. In some embodiments, the storage device 130 may store dataobtained from the imaging device 110, the terminal(s) 140 and/or theprocessing device 120. In some embodiments, the storage device 130 maystore data and/or instructions that the processing device 120 mayexecute or use to perform exemplary methods/systems described in thepresent disclosure. In some embodiments, the storage device 130 mayinclude a mass storage device, a removable storage device, a volatileread-and-write memory, a read-only memory (ROM), or the like, or anycombination thereof. Exemplary mass storage devices may include amagnetic disk, an optical disk, a solid-state drive, etc. Exemplaryremovable storage devices may include a flash drive, a floppy disk, anoptical disk, a memory card, a zip disk, a magnetic tape, etc. Exemplaryvolatile read-and-write memories may include a random access memory(RAM). Exemplary RAM may include a dynamic RAM (DRAM), a double daterate synchronous dynamic RAM (DDR SDRAM), a static RAM (SRAM), athyristor RAM (T-RAM), and a zero-capacitor RAM (Z-RAM), etc. ExemplaryROM may include a mask ROM (MROM), a programmable ROM (PROM), anerasable programmable ROM (EPROM), an electrically erasable programmableROM (EEPROM), a compact disk ROM (CD-ROM), and a digital versatile diskROM, etc. In some embodiments, the storage device 130 may be implementedon a cloud platform. Merely by way of example, the cloud platform mayinclude a private cloud, a public cloud, a hybrid cloud, a communitycloud, a distributed cloud, an inter-cloud, a multi-cloud, or the like,or any combination thereof.

In some embodiments, the storage device 130 may be connected to thenetwork 150 to communicate with one or more other components (e.g., theprocessing device 120, the terminal(s) 140, etc.) in the imageprocessing system 100. One or more components in the image processingsystem 100 may access the data or instructions stored in the storagedevice 130 via the network 150. In some embodiments, the storage device130 may be directly connected to or communicate with one or more othercomponents (e.g., the processing device 120, the terminal(s) 140, etc.)in the image processing system 100. In some embodiments, the storagedevice 130 may be part of the processing device 120.

In some embodiments, a user (e.g., a doctor, or an operator) may operatethe image processing system 100 through the terminal(s) 140. Theterminal(s) 140 may include a mobile device 140-1, a tablet computer140-2, a laptop computer 140-3, or the like, or any combination thereof.In some embodiments, the mobile device 140-1 may include a smart homedevice, a wearable device, a mobile device, a virtual reality device, anaugmented reality device, or the like, or any combination thereof. Insome embodiments, the smart home device may include a smart lightingdevice, a control device of an intelligent electrical apparatus, a smartmonitoring device, a smart television, a smart video camera, aninterphone, or the like, or any combination thereof. In someembodiments, the wearable device may include a bracelet, footgear,eyeglasses, a helmet, a watch, clothing, a backpack, a smart accessory,or the like, or any combination thereof. In some embodiments, the mobiledevice may include a mobile phone, a personal digital assistant (PDA), agaming device, a navigation device, a point of sale (POS) device, alaptop, a tablet computer, a desktop, or the like, or any combinationthereof. In some embodiments, the virtual reality device and/or theaugmented reality device may include a virtual reality helmet, virtualreality glasses, a virtual reality patch, an augmented reality helmet,augmented reality glasses, an augmented reality patch, or the like, orany combination thereof. For example, the virtual reality device and/orthe augmented reality device may include a Google Glass™, an OculusRift™, a Hololens™, a Gear VR™, etc. In some embodiments, theterminal(s) 140 may be part of the processing device 120.

The network 150 may include any suitable network that can facilitate theexchange of information and/or data for the image processing system 100.In some embodiments, one or more components (e.g., the imaging device110, the processing device 120, the storage device 130, the terminal(s)140, etc.) of the image processing system 100 may communicateinformation and/or data with one or more other components of the imageprocessing system 100 via the network 150. For example, the processingdevice 120 may obtain data from the imaging device 110 via the network150. As another example, the processing device 120 may obtain userinstructions from the terminal(s) 140 via the network 150. The network150 may be and/or include a public network (e.g., the Internet), aprivate network (e.g., a local area network (LAN), a wide area network(WAN)), etc.), a wired network (e.g., an Ethernet network), a wirelessnetwork (e.g., an 802.11 network, a Wi-Fi network, etc.), a cellularnetwork (e.g., a Long Term Evolution (LTE) network), a frame relaynetwork, a virtual private network (“VPN”), a satellite network, atelephone network, routers, hubs, switches, server computers, and/or anycombination thereof. Merely by way of example, the network 150 mayinclude a cable network, a wireline network, a fiber-optic network, atelecommunications network, an intranet, a wireless local area network(WLAN), a metropolitan area network (MAN), a public telephone switchednetwork (PSTN), a Bluetooth™ network, a ZigBee™ network, a near fieldcommunication (NFC) network, or the like, or any combination thereof. Insome embodiments, the network 150 may include one or more network accesspoints. For example, the network 150 may include wired and/or wirelessnetwork access points, such as base stations and/or internet exchangepoints, through which one or more components of the image processingsystem 100 may be connected to the network 150 to exchange data and/orinformation.

It should be noted that the above description of the image processingsystem 100 is merely provided for the purposes of illustration, and notintended to limit the scope of the present disclosure. For personshaving ordinary skills in the art, multiple variations and modificationsmay be made under the teachings of the present disclosure. For example,the assembly and/or function of the image processing system 100 may bevaried or changed according to specific implementation scenarios.

FIG. 2 is a schematic diagram illustrating exemplary hardware and/orsoftware components of an exemplary computing device according to someembodiments of the present disclosure. The computing device 200 may beused to implement any component of the image processing system 100 asdescribed herein. For example, the processing device 120 and/or theterminal(s) 140 may be implemented on the computing device 200,respectively, via its hardware, software program, firmware, or acombination thereof. Although only one such computing device is shown,for convenience, the computer functions relating to the image processingsystem 100 as described herein may be implemented in a distributedmanner on a number of similar platforms, to distribute the processingload.

As shown in FIG. 2, the computing device 200 may include a processor210, a storage 220, an input/output (I/O) 230, and a communication port240.

The processor 210 may execute computer instructions (e.g., programcodes) and perform functions of the image processing system 100 (e.g.,the processing device 120) in accordance with techniques describedherein. The computer instructions may include, for example, routines,programs, objects, components, signals, data structures, procedures,modules, and functions, which perform particular functions describedherein. For example, the processor 210 may process data obtained fromthe imaging device 110, the terminal(s) 140, the storage device 130,and/or any other component of the image processing system 100.Specifically, the processor 210 may process one or more measured datasets obtained from the imaging device 110. For example, the processor210 may generate an image based on the data set(s). In some embodiments,the generated image may be stored in the storage device 130, the storage220, etc. In some embodiments, the generated image may be displayed on adisplay device by the I/O 230. In some embodiments, the processor 210may perform instructions obtained from the terminal(s) 140. In someembodiments, the processor 210 may include one or more hardwareprocessors, such as a microcontroller, a microprocessor, a reducedinstruction set computer (RISC), an application-specific integratedcircuits (ASICs), an application-specific instruction-set processor(ASIP), a central processing unit (CPU), a graphics processing unit(GPU), a physics processing unit (PPU), a microcontroller unit, adigital signal processor (DSP), a field-programmable gate array (FPGA),an advanced RISC machine (ARM), a programmable logic device (PLD), anycircuit or processor capable of executing one or more functions, or thelike, or any combinations thereof.

Merely for illustration, only one processor is described in thecomputing device 200. However, it should be noted that the computingdevice 200 in the present disclosure may also include multipleprocessors. Thus operations and/or method steps that are performed byone processor as described in the present disclosure may also be jointlyor separately performed by the multiple processors. For example, if inthe present disclosure the processor of the computing device 200executes both operation A and operation B, it should be understood thatoperation A and operation B may also be performed by two or moredifferent processors jointly or separately in the computing device 200(e.g., a first processor executes operation A and a second processorexecutes operation B, or the first and second processors jointly executeoperations A and B).

The storage 220 may store data/information obtained from the imagingdevice 110, the terminal(s) 140, the storage device 130, or any othercomponent of the image processing system 100. In some embodiments, thestorage 220 may include a mass storage device, a removable storagedevice, a volatile read-and-write memory, a read-only memory (ROM), orthe like, or any combination thereof. For example, the mass storagedevice may include a magnetic disk, an optical disk, a solid-statedrive, etc. The removable storage device may include a flash drive, afloppy disk, an optical disk, a memory card, a zip disk, a magnetictape, etc. The volatile read-and-write memory may include a randomaccess memory (RAM). The RAM may include a dynamic RAM (DRAM), a doubledate rate synchronous dynamic RAM (DDR SDRAM), a static RAM (SRAM), athyristor RAM (T-RAM), and a zero-capacitor RAM (Z-RAM), etc. The ROMmay include a mask ROM (MROM), a programmable ROM (PROM), an erasableprogrammable ROM (PEROM), an electrically erasable programmable ROM(EEPROM), a compact disk ROM (CD-ROM), and a digital versatile disk ROM,etc. In some embodiments, the storage 220 may store one or more programsand/or instructions to perform exemplary methods described in thepresent disclosure. For example, the storage 220 may store a program forgenerating a target image associated with a target object.

The I/O 230 may input and/or output signals, data, and/or information,etc. In some embodiments, the I/O 230 may enable user interaction withthe image processing system 100 (e.g., the processing device 120). Insome embodiments, the I/O 230 may include an input device and an outputdevice. Exemplary input devices may include a keyboard, a mouse, a touchscreen, a microphone, or the like, or a combination thereof. Exemplaryoutput devices may include a display device, a loudspeaker, a printer, aprojector, or the like, or a combination thereof. Exemplary displaydevices may include a liquid crystal display (LCD), a light-emittingdiode (LED)-based display, a flat panel display, a curved screen, atelevision device, a cathode ray tube (CRT), or the like, or acombination thereof.

The communication port 240 may be connected with a network (e.g., thenetwork 150) to facilitate data communications. The communication port240 may establish connections between the processing device 120 and theimaging device 110, the terminal(s) 140, or the storage device 130. Theconnection may be a wired connection, a wireless connection, or acombination of both that enables data transmission and reception. Thewired connection may include an electrical cable, an optical cable, atelephone wire, or the like, or any combination thereof. The wirelessconnection may include a Bluetooth network, a Wi-Fi network, a WiMaxnetwork, a WLAN, a ZigBee network, a mobile network (e.g., 3G, 4G, 5G,etc.), or the like, or any combination thereof. In some embodiments, thecommunication port 240 may be a standardized communication port, such asRS232, RS485, etc. In some embodiments, the communication port 240 maybe a specially designed communication port. For example, thecommunication port 240 may be designed in accordance with the digitalimaging and communications in medicine (DICOM) protocol.

FIG. 3 is a schematic diagram illustrating exemplary hardware and/orsoftware components of an exemplary mobile device according to someembodiments of the present disclosure. As illustrated in FIG. 3, themobile device 300 may include a communication platform 310, a display320, a graphics processing unit (GPU) 330, a central processing unit(CPU) 340, an I/O 350, a memory 360, and a storage 390. In someembodiments, any other suitable component, including but not limited toa system bus or a controller (not shown), may also be included in themobile device 300. In some embodiments, an operating system 370 (e.g.,iOS™, Android™, Windows Phone™, Harmony OS, etc.) and one or moreapplications (Apps) 380 may be loaded into the memory 360 from thestorage 390 in order to be executed by the CPU 340. The applications 380may include a browser or any other suitable mobile apps for receivingand rendering information relating to image processing or otherinformation from the processing device 120. User interactions with theinformation stream may be achieved via the I/O 350 and provided to theprocessing device 120 and/or other components of the image processingsystem 100 via the network 150. In some embodiments, a user may inputparameters to the image processing system 100, via the mobile device300.

In order to implement various modules, units and their functionsdescribed above, a computer hardware platform may be used as hardwareplatforms of one or more elements (e.g., the processing device 120and/or other components of the image processing system 100 described inFIG. 1). Since these hardware elements, operating systems and programlanguages are common; it may be assumed that persons skilled in the artmay be familiar with these techniques and they may be able to provideinformation needed in the imaging and assessing according to thetechniques described in the present disclosure. A computer with the userinterface may be used as a personal computer (PC), or other types ofworkstations or terminal devices. After being properly programmed, acomputer with the user interface may be used as a server. It may beconsidered that those skilled in the art may also be familiar with suchstructures, programs, or general operations of this type of computingdevice.

FIG. 4A is a block diagram illustrating an exemplary processing deviceaccording to some embodiments of the present disclosure. In someembodiments, the processing device 120 a may be configured to segmentone or more target objects (e.g., blood vessels) in an initial imagefrom the background (e.g., regions including interfering objects) of theinitial image. The processing device 120 a may be implemented assoftware and/or hardware. In some embodiments, the processing device 120a may be part of an imaging device (e.g., the imaging device 110). Moredescriptions about the imaging device may be found elsewhere in thepresent disclosure (e.g., FIG. 1 and the descriptions thereof).

As illustrated in FIG. 4A, the processing device 120 a may include anobtaining module 4510, and a segmentation module 4520.

Each of the modules may be a hardware circuit that is designed toperform certain actions, e.g., according to a set of instructions storedin one or more storage media, and/or any combination of the hardwarecircuit and one or more storage media.

The obtaining module 4510 may be configured to obtain an initial imageand/or an intermediate image corresponding to the initial image.

The segmentation module 4520 may be configured to input the initialimage and the intermediate image into a trained processing model togenerate a target image associated with one or more target objects.

In some embodiments, the initial image and the intermediate image may beobtained by the obtaining module 4510, and the initial image and theintermediate image may be inputted into the trained processing model bythe segmentation module 4520. As a result, the target image may begenerated. In comparison with traditional segmentation techniques (e.g.,blood vessel segmentation techniques) that are not suitable for thesegmentation of target objects with relatively complicated structures(e.g., coronary arteries), the trained processing model may be used tosegment target objects more precisely, and accordingly, the segmentationaccuracy of the target objects may be improved.

In some embodiments, the segmentation module 4520 may include a fusionunit (not shown). The fusion unit may be configured to fuse the initialimage and the intermediate image to generate a fusion image and inputthe fusion image into the trained processing model.

Alternatively, and/or additionally, the fusion unit may further beconfigured to input in a parallel mode the initial image and theintermediate image into two different input channels of the trainedprocessing model, and input the initial image and the intermediate imageinto the trained processing model through the corresponding inputchannels, respectively.

Alternatively, and/or additionally, the fusion unit may further beconfigured to determine the fusion image based on a sum or product ofeach pixel or voxel of the initial image and a corresponding pixel orvoxel of the intermediate image, and input the fusion image into thetrained processing model.

In some embodiments, the obtaining module 4510 may include an extractionunit (not shown). The extraction unit may be configured to process theinitial image using one or more filters to generate the intermediateimage.

Alternatively, and/or additionally, the extraction unit may further beconfigured to determine a Hessian matrix corresponding to each pixel orvoxel of the initial image, and determine at least one characteristicvalue corresponding to the each pixel or voxel based on the Hessianmatrix. The extraction unit may determine a response value correspondingto the each pixel or voxel by enhancing the at least one characteristicvalue corresponding to the each pixel or voxel using a first filter(e.g., a tubular filter or a linear filter). The extraction unit maygenerate the intermediate image based on a plurality of response valuescorresponding to a plurality of pixels or voxels of the initial image.

Alternatively, and/or additionally, the extraction unit may further beconfigured to smooth, before determining the Hessian matrixcorresponding to the each pixel or voxel of the initial image, theinitial image using one or more second filters (e.g., Gaussian filterswith kernels of different sizes) to generate one or more smoothedinitial images. The one or more smoothed initial images may be processedto generate the intermediate image.

In some embodiments, the extraction unit may be configured to select amaximum response value corresponding to each pixel or voxel of theinitial image among one or more response values corresponding to pixelsor voxels at a same position in the one or more smoothed initial images.The extraction unit may generate the intermediate image based on aplurality of maximum response values corresponding to a plurality ofpixels or voxels of the initial image.

In some embodiments, the processing device 120 a may further include atraining module 4530. The training module may be configured to train aninitial processing model to generate the trained processing model beforethe usage of the trained processing model. More descriptions about thetraining module may be found elsewhere in the present disclosure (e.g.,FIG. 4B and the descriptions thereof).

In some embodiments, the processing device 120 a may further include anextraction module (not shown) configured to extract a largest connecteddomain in the target image as an updated target image after generatingthe target image.

It should be noted that the above descriptions about the processingdevice 120 a are merely provided for the purposes of illustration, andnot intended to limit the scope of the present disclosure. For personshaving ordinary skills in the art, multiple variations and modificationsmay be made under the teachings of the present disclosure. However,those variations and modifications do not depart from the scope of thepresent disclosure. For example, some other components/modules (e.g., astorage module) may be added into the processing device 120 a. In someembodiments, the training module 4530 may be omitted. In someembodiments, the training module 4530 may be implemented on anotherprocessing device 120 b as illustrated in FIG. 4B.

FIG. 4B is a block diagram illustrating another exemplary processingdevice according to some embodiments of the present disclosure. Asillustrated in FIG. 4B, the processing device 120 b may include aninitial sample image obtaining unit 410, an intermediate sample imagedetermination unit 420, a predicted image determination unit 430, and aparameter adjustment unit 440. Each of the modules described above maybe a hardware circuit that is designed to perform certain actions, e.g.,according to a set of instructions stored in one or more storage media,and/or any combination of the hardware circuit and one or more storagemedia.

The initial sample image obtaining unit 410 may be configured to obtaina plurality of initial sample images.

The intermediate sample image determination unit 420 may be configuredto generate a plurality of intermediate sample images based on theplurality of initial sample images. Each of the plurality ofintermediate sample images may correspond to an initial sample image.

The predicted image determination unit 430 may be configured to inputeach initial sample image and a corresponding intermediate sample imageinto an initial processing model to generate a predicted imagecorresponding to the initial sample image.

The parameter adjustment unit 440 may be configured to adjust and/orupdate one or more parameters of the processing model based on adifference between each predicted image and a corresponding desiredtarget image.

In some embodiments, the predicted image determination unit 430 mayinclude an image fusion sub-unit (not shown). The image fusion sub-unitmay be configured to fuse each initial sample image and a correspondingintermediate sample image to generate a fusion sample image and inputthe fusion sample image into the processing model.

Alternatively, and/or additionally, the image fusion sub-unit mayfurther be configured to input each initial sample image and thecorresponding intermediate sample image into two different inputchannels of the processing model, and input the images into theprocessing model.

In some embodiments, the image fusion sub-unit may further be configuredto generate the fusion sample image based on a sum or product of eachpixel or voxel of the initial sample image and a corresponding pixel orvoxel of the intermediate sample image, and input the fusion sampleimage into the processing model.

In some embodiments, the intermediate sample image determination unit420 may include a filtering sub-unit (not shown). The filtering sub-unitmay be configured to process an initial sample image using one or morefilters to generate an intermediate sample image.

In some embodiments, the filtering sub-unit may further be configured todetermine at least one characteristic value of a Hessian matrixcorresponding to each pixel or voxel of the initial sample image. Thefiltering sub-unit may determine a response value corresponding to eachpixel or voxel of the initial sample image by enhancing the at least onecharacteristic value corresponding to each pixel or voxel of the initialsample image using a first filter (e.g., a tubular filter, or a linearfilter). The filtering sub-unit may generate the intermediate sampleimage based on a plurality of response values corresponding to aplurality of pixels or voxels of the initial sample image.

Alternatively and/or additionally, before determining the Hessian matrixcorresponding to the each pixel or voxel of the initial sample image,the filtering sub-unit may further be configured to smooth the initialsample image using one or more second filters (e.g., Gaussian filterswith kernels of different sizes) to generate one or more smoothedinitial sample images. The one or more smoothed initial sample imagesmay be processed to generate the intermediate sample image.

In some embodiments, the one or more second filters may include aGaussian filter, a tubular filter, a linear filter, a Wiener filter, orthe like, or any combination thereof.

Alternatively, and/or additionally, the filtering sub-unit may furtherbe configured to select a maximum response value corresponding to eachpixel or voxel of the initial sample image among one or more responsevalues corresponding to pixels or voxels at a same position in the oneor more smoothed initial sample images. The filtering sub-unit maygenerate the intermediate sample image based on a plurality of maximumresponse values corresponding to a plurality of pixels or voxels of theinitial sample image.

In some embodiments, the processing model may include a convolutionalneural network (CNN) model (e.g., a U-Net neural network model, or aV-Net neural network model), a deep CNN (DCNN) model, a fullyconvolutional network (FCN) model, a recurrent neural network (RNN)model, or the like, or any combination thereof.

In some embodiments, the processing device 120 a or 120 b may executeany operation of the present disclosure, and may include one or moreother modules for performing a corresponding function in the operations.

It should be noted that the above descriptions about the processingdevice 120 b are merely provided for the purposes of illustration, andnot intended to limit the scope of the present disclosure. For personshaving ordinary skills in the art, multiple variations and modificationsmay be made under the teachings of the present disclosure. However,those variations and modifications do not depart from the scope of thepresent disclosure. For example, the initial sample image obtaining unit410 and the intermediate sample image determination unit 420 may beintegrated into a single module. As another example, some othercomponents/modules (e.g., a storage unit) may be added into theprocessing device 120 b.

FIG. 5 is a flowchart illustrating an exemplary process for generating atarget image associated with one or more target objects according tosome embodiments of the present disclosure.

In some embodiments, the process 500 may be implemented as a set ofinstructions (e.g., an application) stored in the storage device 130,storage 220, or storage 390. The processing device 120, the processor210 and/or the CPU 340 may execute the set of instructions, and whenexecuting the instructions, the processing device 120, the processor 210and/or the CPU 340 may be configured to perform the process 500. Theoperations of the illustrated process presented below are intended to beillustrative. In some embodiments, the process 500 may be accomplishedwith one or more additional operations not described and/or without oneor more of the operations discussed. Additionally, the order of theoperations of the process 500 illustrated in FIG. 5 and described belowis not intended to be limiting.

In some embodiments, the process 500 may be performed to separate one ormore target objects (e.g., blood vessels) in an initial image from abackground of the initial image. In some embodiments, the process 500may be executed by a target object segmentation device implemented assoftware and/or hardware. In some embodiments, the target objectsegmentation device may be part of an imaging device (e.g., the imagingdevice 110). More descriptions about the imaging device may be foundelsewhere in the present disclosure (e.g., FIG. 1 and the descriptionsthereof).

In some embodiments, the one or more target objects may have the same orsimilar shapes. An object having a shape substantially different fromthat of the one or more target objects may be referred to as aninterfering object. For example, if each of the one or more targetobjects has a linear shape, then the interfering objects may have a dotshape, a plane shape, a globular shape, or the like. As another example,if each of the one or more target objects has a dot shape, then theinterfering objects may have a linear shape, a plane shape, a globularshape, or the like. In some embodiments, the one or more target objectsmay have the same or similar grey levels. In some embodiments, the greylevels of the target objects may be different from the grey levels ofthe interfering objects. The background of the initial image may referto one or more regions including representations of the interferingobjects in the initial image. For example, for a heart CT image, if theone or more target objects are coronary arteries, regions includingrepresentations of the cardiac muscles, the spleen, the lungs, thethorax, the stomach, bones, or the like, in the heart CT image may beregarded as the background.

In some embodiments, segmenting the one or more target objects from theinitial image (or separating the target objects from the background ofthe initial image) may refer to generating a target image associatedwith (or including representations of) the one or more target objects.In some embodiments, the target object segmentation device may bereferred to as a processing device (e.g., the processing device 120). Insome embodiments, the target object segmentation device may be part ofthe processing device 120.

In some embodiments, as shown in FIG. 5, an exemplary target imagegeneration process is provided. The process 500 may include one or moreof the following operations.

In 5110, an initial image may be obtained. In 5115, an intermediateimage corresponding to the initial image may be obtained.

In some embodiments, the processing device 120 a (e.g., the obtainingmodule 4510) may perform operation 5110. In some embodiments, theprocessing device 120 a (e.g., the obtaining module 4510) may performoperation 5115.

In some embodiments, the initial image may be generated based on imagingdata (e.g., projection data) acquired by the imaging device 110. Forexample, the imaging data may be acquired by scanning a subject withinthe detection region of the imaging device 110. In some embodiments, theinitial image may be a CT image, an MR image, a DSA image, a PET image,a SPECT image, a DR image, or the like. The intermediate image may begenerated by processing the initial image. For example, the intermediateimage may be generated by extracting or segmenting one or more targetobjects from the initial image. The intermediate image may include acoarse representation of the one or more target objects (or at least aportion thereof).

In some embodiments, the subject may include a patient or a portionthereof (e.g., the heart, the head, the neck, the thorax, the stomach,soft tissue, a tumor, nodules, etc.) as described elsewhere in thepresent disclosure (e.g., FIG. 1 and the descriptions thereof). Thesubject may include one or more target objects. An exemplary targetobject may include a blood vessel (e.g., a blood vessel of a heart), anodule in an organ (e.g., a liver, a kidney), etc. In some embodiments,the initial image may be generated based on imaging data acquired aftera contrast agent is injected into blood vessels, and the target objectsmay include the blood vessels. In some embodiments, the initial imagemay include a plurality of first pixels or voxels representing thesubject. In some embodiments, the first pixels or voxels may havecorresponding pixel/voxel values or characteristics (e.g., luminancevalues, grey values, colors (e.g., RGB values), saturation values,etc.).

The intermediate image may include a plurality of second pixels orvoxels associated with at least a portion of the one or more targetobjects in the initial image. In some embodiments, a second pixel orvoxel in the intermediate image may correspond to a first pixel or voxelin the initial image. That is, a second pixel or voxel in theintermediate image and a corresponding first pixel or voxel in theinitial image may represent a same physical portion or position of thesubject. In some embodiments, the intermediate image (including a coarserepresentation of the target objects) may include less details than theinitial image or a target image that includes a fine representation ofthe target objects. In comparison with the intermediate image, thetarget image may include a precise and/or accurate representation of thetarget objects. For example, the target image may include relativelycomplete information of the one or more target objects, while theintermediate image may include less information of the target objects ormay further include, in addition to the representation of the targetobjects, representations of one or more interfering objects. As anotherexample, if the initial image is a heart image (e.g., as shown in FIG.11A), and the target objects include the coronary artery, then theintermediate image (e.g., as shown in FIG. 11B) may include a coarserepresentation of a portion of the coronary artery, a portion ofinterfering objects (e.g., one or more blood vessels other than thecoronary artery, a trachea, etc.), while the target image may onlyinclude a fine representation of the coronary artery. In someembodiments, the intermediate image may include a skeleton or aframework (or a portion thereof) of the target objects.

In some embodiments, the one or more target objects (e.g., the bloodvessels) may be extracted or segmented from the initial image togenerate the intermediate image based on one or more segmentationalgorithms (e.g., a threshold segmentation algorithm), one or morefilters, or the like, or any combination thereof. Exemplary filters mayinclude a Gaussian filter, a linear filter, a tubular filter (or acoaxial filter), a Wiener filter, a threshold filter, or the like, orany combination thereof.

Exemplary segmentation algorithms may include a threshold segmentationalgorithm, a region growing segmentation algorithm, an energy-based 3Dreconstruction segmentation algorithm, a level set-based segmentationalgorithm, a region split and/or merge segmentation algorithm, an edgetracking segmentation algorithm, a statistical pattern recognitionalgorithm, a C-means clustering segmentation algorithm, a deformablemodel segmentation algorithm, a graph search segmentation algorithm, aneural network segmentation algorithm, a geodesic minimal pathsegmentation algorithm, a target tracking segmentation algorithm, anatlas-based segmentation algorithm, a rule-based segmentation algorithm,a coupled surface segmentation algorithm, a model-based segmentationalgorithm, a deformable organism segmentation algorithm, a modelmatching algorithm, an artificial intelligence algorithm, or the like,or any combination thereof.

In some embodiments, to generate the intermediate image efficiently, theprocessing device 120 may remove at least a portion of backgroundinformation from the initial image. For example, for a heart CT image,if the target objects are blood vessels, the processing device 120 mayremove at least a portion of the representations of the cardiac muscles,the spleen, the thorax, the stomach, the lungs, bones, or the like. Asanother example, the processing device 120 may remove at least a portionof the representations of organs far away from the target objects. Theprocessing device 120 may reserve representations of regions similar toor the same as the blood vessels.

In some embodiments, a Hessian matrix corresponding to each first pixelor voxel of the initial image may be determined. At least onecharacteristic value corresponding to the each first pixel of voxel maybe determined based on the Hessian matrix. A response valuecorresponding to the each first pixel or voxel may be determined byenhancing the at least one characteristic value corresponding to theeach first pixel or voxel using a first filter (e.g., if the one or moretarget objects have a linear shape, the first filter may be a tubularfilter). The intermediate image may be generated based on a plurality ofresponse values corresponding to the plurality of first pixels or voxelsof the initial image. In some embodiments, before determining the atleast one characteristic value of the Hessian matrix corresponding tothe each first pixel or voxel, the initial image may be smoothed using asecond filter (e.g., a Gaussian filter). The first filter may be thesame as or different from the second filter. For example, the firstfilter may be a tubular filter, while the second filter may be aGaussian filter. As another example, both the first filter and thesecond filter may be tubular filters. More descriptions for determiningthe intermediate image based on the Hessian matrix may be foundelsewhere in the present disclosure (e.g., FIG. 8 and the descriptionsthereof).

In some embodiments, the one or more filters used for processing theinitial image may be determined based on a shape of the target objects.For example, if the target objects have a dot shape (e.g., dot nodules),a dot filter may be used for processing the initial image. As anotherexample, if the target objects have a linear shape (e.g., bloodvessels), a tubular filter or a line filter may be used for processingthe initial image. In some embodiments, blood vessel segmentation isdescribed for illustration purposes, and a tubular filter is taken as anexample in the present disclosure.

In some embodiments, the initial image may be obtained from the imagingdevice (e.g., the imaging device 110), the storage device 130, or anyother storage device (e.g., a cloud platform). For example, the initialimage may be generated by a processing device (e.g., the processingdevice 120, or a processing device other than the processing device120), and stored in the imaging device, the storage device 130, or anyother storage device. In some embodiments, the processing device 120 mayobtain the initial image from the imaging device, the storage device130, or any other storage device directly. In some embodiments, theimaging device may transmit acquired imaging data (e.g., projectiondata) to the storage device 130, or any other storage device forstorage, and the processing device 120 may obtain the imaging data fromthe storage device 130, or any other storage device, and generate theinitial image based on the imaging data.

In some embodiments, the intermediate image may be obtained from theimaging device (e.g., the imaging device 110), the storage device 130,or any other storage device (e.g., the cloud platform). For example, theintermediate image may be generated based on the initial image by aprocessing device (e.g., the processing device 120, or a processingdevice other than the processing device 120), and stored in the imagingdevice (e.g., the imaging device 110), the storage device 130, or anyother storage device. In some embodiments, the processing device 120 mayobtain the intermediate image from the imaging device, the storagedevice 130, or any other storage device directly. In some embodiments,the processing device 120 may obtain the intermediate image byprocessing the initial image.

In 5120, a target image associated with a target object may begenerated. In some embodiments, the initial image and the intermediateimage may be inputted into a trained processing model to generate thetarget image.

In some embodiments, the processing device 120 a (e.g., the segmentationmodule 4520) may perform operation 5120.

In some embodiments, the trained processing model may be obtained bytraining an initial processing model based on a plurality of trainingsamples. In some embodiments, the processing model may include aconvolutional neural network (CNN) model (e.g., a U-Net neural networkmodel, or a V-Net neural network model), or the like, or any combinationthereof.

The trained processing model may be configured to segment the one ormore target objects from the initial image based on the intermediateimage. In some embodiments, the trained processing model may include acoarse segmentation network and/or a fine segmentation network. Eachsegmentation network may correspond to a stage of the trained processingmodel. The coarse segmentation network may be configured to roughlydetermine locations of the one or more target objects in the initialimage and generate a preliminary segmentation image of the one or moretarget objects. In some embodiments, the initial image may be downsampled (e.g., from a relatively high original resolution (e.g., 0.5 mm)to a relatively low resolution (e.g., 1.6 mm)) and inputted into thecoarse segmentation network, and the preliminary segmentation image maybe outputted from the coarse segmentation network. Accordingly, in thesubsequent processing, the trained processing model may not processregions beyond the one or more target objects, thereby increasing asegmentation speed of the one or more target objects. The finesegmentation network may be configured to determine a fine image (i.e.,the target image) of the one or more target objects. In someembodiments, one or more extracted region(s) (with the originalresolution) in the initial image corresponding to the target object(s)in the preliminary segmentation image may be determined. In someembodiments, the extracted region(s) (or pixels or voxels thereof) andthe intermediate image may be inputted into the fine segmentationnetwork to generate the fine image (i.e., the target image) of the oneor more target objects. In some embodiments, the coarse segmentationnetwork and the fine segmentation network (i.e., the trained processingmodel) may be determined by training the initial processing model usingthe plurality of training samples in a training process.

The training process may be performed by the processing device 120 or aprocessing device different from the processing device 120. The trainedprocessing model may be stored in a storage device (e.g., the storagedevice 130, the storage 220, or the storage 390). The processing modelmay be trained online of offline. In some embodiments, the processingmodel may be trained in an external device, and stored in a storagedevice (e.g., the storage device 130, the storage 220, the storage 390,or an external storage device that can communicate with the imageprocessing system 100 (e.g., via the network 150)). In some embodiments,the processing device 120 a (e.g., the segmentation module 4520) mayretrieve the trained processing model from the storage device (e.g., inresponse to receipt of a request for image segmentation). Moredescriptions regarding the determination of the trained processing modelmay be found elsewhere in the present disclosure (e.g., FIG. 7B and thedescriptions thereof).

After the initial image and the intermediate image corresponding to theinitial image are obtained, the initial image and the intermediate imagemay be inputted into the trained processing model, and the target imagemay be generated.

In some embodiments, the target image may include a representation ofthe one or more target objects without the background. In someembodiments, the processing device 120 may transmit the target image toa terminal (e.g., the terminal(s) 140 in the image processing system100).

In some embodiments, after the target image is generated, a largestconnected domain in the target image may be extracted as an updatedtarget image. Therefore, the representation of the one or more targetobjects may be reserved, while representations of the interferingobjects may be deleted, thereby improving the segmentation accuracy ofthe target objects.

As described in some embodiments of the present disclosure, the targetimage may be obtained by obtaining the initial image and theintermediate image corresponding to the initial image, and inputting theinitial image and the intermediate image into the trained processingmodel. In comparison with traditional segmentation techniques (e.g.,blood vessel segmentation techniques) that are not suitable for thesegmentation of target objects with relatively complicated structures(e.g., coronary arteries), operations illustrated in the process 500 maybe performed to segment target objects more precisely, and accordingly,the segmentation accuracy of the target objects using the trainedprocessing model may be improved.

It should be noted that the above descriptions are merely provided forthe purposes of illustration, and not intended to limit the scope of thepresent disclosure. For persons having ordinary skills in the art,multiple variations and modifications may be made under the teachings ofthe present disclosure. However, those variations and modifications donot depart from the scope of the present disclosure. In someembodiments, one or more operations may be omitted and/or one or moreadditional operations may be added. For example, operations 5110 and5115 may be integrated into a single operation. As another example, oneor more other optional operations (e.g., a storing operation) may beadded elsewhere in the process 500. In the storing operation, theprocessing device 120 may store information and/or data (e.g., theinitial image, the intermediate image, the trained processing model,etc.) associated with the image processing system 100 in a storagedevice (e.g., the storage device 130) disclosed elsewhere in the presentdisclosure.

FIG. 6 is a flowchart illustrating an exemplary process for generating atarget image associated with one or more target objects according tosome embodiments of the present disclosure. The process 600 may includemore descriptions of one or more operations in FIG. 5.

In some embodiments, the process 600 may be implemented as a set ofinstructions (e.g., an application) stored in the storage device 130,storage 220, or storage 390. The processing device 120, the processor210 and/or the CPU 340 may execute the set of instructions, and whenexecuting the instructions, the processing device 120, the processor 210and/or the CPU 340 may be configured to perform the process 600. Theoperations of the illustrated process presented below are intended to beillustrative. In some embodiments, the process 600 may be accomplishedwith one or more additional operations not described and/or without oneor more of the operations discussed. Additionally, the order of theoperations of the process 600 illustrated in FIG. 6 and described belowis not intended to be limiting. In some embodiments, an operationillustrated in FIG. 5 may be performed according to one or moreoperations of the process 600. For example, operation 5115 in FIG. 5 maybe performed according to operation 6220. As another example, operation5120 in FIG. 5 may be performed according to operations 6225-6230.

Specifically, in some embodiments, the operation 5120 “generating atarget image associated with a target object using a trained processingmodel” may be illustrated as the operation 6225 “fusing the initialimage and the intermediate image to obtain a fusion image,” and theoperation 6230 “inputting the fusion image into a trained processingmodel” to clarify the application mechanism of the trained processingmodel.

Specifically, in some embodiments, the operation 5115 “obtaining anintermediate image corresponding to the initial image” may beillustrated as the operation 6220 “generating an intermediate image byprocessing the initial image using one or more filters” to clarify thedetermination mechanism of the intermediate image.

In some embodiments, as shown in FIG. 6, an exemplary target imagegeneration process is provided. The process 600 may include one or moreof the following operations.

In 6210, an initial image may be obtained.

In some embodiments, the processing device 120 a (e.g., the obtainingmodule 4510) may perform operation 6210. The initial image may include aplurality of first pixels or voxels representing a subject. The subjectmay include one or more target objects (e.g., one or more blood vessels(e.g., coronary arteries)). Operation 6210 may be similar to operation5110. More descriptions regarding the initial image may be foundelsewhere in the present disclosure (e.g., FIG. 5 and descriptionsthereof).

In 6220, an intermediate image may be generated by processing theinitial image using one or more filters.

In some embodiments, the processing device 120 a (e.g., the obtainingmodule 4510) may perform operation 6220. The intermediate image mayinclude a plurality of second pixels or voxels associated with at leasta portion of the one or more target objects in the initial image. Moredescriptions of the intermediate image may be found elsewhere in thepresent disclosure (e.g., FIG. 5 and descriptions thereof).

In some embodiments, the one or more filters may include a Gaussianfilter, a tubular filter, a linear filter, a Wiener filter, a thresholdfilter, or the like, or any combination thereof.

In some embodiments, to generate the intermediate image by processingthe initial image using the one or more filters, a Hessian matrixcorresponding to each first pixel or voxel of the initial image may bedetermined. At least one characteristic value corresponding to the eachfirst pixel of voxel may be determined based on the Hessian matrix. Insome embodiments, a response value corresponding to the each first pixelor voxel may be determined by processing the at least one characteristicvalue corresponding to the each first pixel or voxel using a firstfilter (e.g., a tubular filter, or a linear filter) of the one or morefilters. In some embodiments, the intermediate image may be generatedbased on a plurality of response values corresponding to the pluralityof first pixels or voxels of the initial image.

In some embodiments, because characteristic values of a Hessian matrixcorresponding to a pixel or voxel of an image may relate to shapefeatures of the pixel or voxel, each first pixel or voxel of the initialimage may be associated with at least one characteristic value of aHessian matrix corresponding to the each first pixel or voxel. In someembodiments, the at least one characteristic value of the Hessian matrixcorresponding to the each first pixel or voxel may be enhanced using atubular filter (or a linear filter). Therefore, pixel or voxel values offirst pixels or voxels representing an object (e.g., a target object)having a tubular shape may be enhanced or increased, while pixel orvoxel values of first pixels or voxels representing other objects (e.g.,interfering objects) having non-tubular shape may be suppressed ordecreased. A response value corresponding to each first pixel or voxelmay be determined by enhancing the at least one characteristic valueusing the tubular filter (or the linear filter), and the intermediateimage may be generated based on the plurality of response valuescorresponding to the plurality of first pixels or voxels of the initialimage. For example, the intermediate image may be generated by combiningthe plurality of response values according to an arrangement order ofthe plurality of first pixels or voxels in the initial image.

It should be noted that, in some embodiments, before determining the atleast one characteristic value of the Hessian matrix corresponding tothe each first pixel or voxel, the initial image may be smoothed using asecond filter (e.g., a Gaussian filter) of the one or more filters toimprove a precision or accuracy of the intermediate image, therebyimproving a segmentation accuracy of the trained processing model. Insome embodiments, the smoothed initial image may be processed insubsequent operations. For example, the intermediate image may begenerated based on the smoothed initial image. The second filter mayinclude a Gaussian filter, a tubular filter, a linear filter, a Wienerfilter, a threshold filter, or the like, or any combination thereof.

In some embodiments, in order to improve the precision and/or accuracyof the intermediate image, in the generation of the intermediate imagebased on the plurality of response values corresponding to the pluralityof first pixels or voxels of the initial image, the initial image may besmoothed using two or more second filters (e.g., Gaussian filters withkernels of different sizes) to generate two or more smoothed initialimages. In some embodiments, a Hessian matrix corresponding to eachpixel or voxel of each of the two or more smoothed initial images may bedetermined. Accordingly, at least one characteristic value correspondingto the each pixel or voxel of the each smoothed initial image may bedetermined. Thus, a response value corresponding to the each pixel orvoxel of the each smoothed initial image may be determined by enhancingthe at least one characteristic value using the first filter of the oneor more filters. Accordingly, two or more response values (the number orcount of which may be the same as the number or count of the two or moresecond filters) corresponding to each first pixel or voxel of theinitial image may be determined. In some embodiments, a maximum responsevalue corresponding to each first pixel or voxel of the initial imagemay be selected among the two or more response values corresponding tothe each first pixel or voxel. That is, a maximum response valuecorresponding to a specific first pixel or voxel of the initial imagemay be selected among the response values corresponding to pixels orvoxels (that are at a same position of the specific first pixel orvoxel) of the two or more smoothed initial images. In some embodiments,the processing device 120 may generate the intermediate image based on aplurality of maximum response values corresponding to a plurality offirst pixels or voxels of the initial image.

Merely by way of example, an initial image may be smoothed by threesecond filters, and accordingly, three response values (e.g., R1, R2,and R3) corresponding to a first pixel may be determined. The processingdevice 120 may determine a maximum response value (e.g., R3) as a targetresponse value corresponding to the first pixel.

In some embodiments, in the generation of the intermediate image basedon the plurality of response values corresponding to the plurality offirst pixels or voxels of the initial image, two or more candidateintermediate images generated based on two or more second filters may bedetermined. A count or number of pixels or voxels (in each of the two ormore candidate intermediate images) whose response values are greaterthan a threshold may be determined. The processing device 120 maydetermine a candidate intermediate image (with the largest count ornumber of pixels or voxels whose response values are greater than thethreshold) as the intermediate image.

Merely by way of example, an initial image may be smoothed by threesecond filters, and accordingly, three smoothed initial images (e.g.,S1, S2, S3) may be generated. A candidate intermediate image may begenerated based on each of the three smoothed initial images, andaccordingly, three candidate intermediate images (e.g., M1(corresponding to S1), M2 (corresponding to S2), M3 (corresponding toS3)) may be generated. A first count or number of pixels or voxels (incandidate intermediate image M1) whose response values are greater thanthe threshold may be determined as C1. A second count or number ofpixels or voxels (in candidate intermediate image M2) whose responsevalues are greater than the threshold may be determined as C2. A thirdcount or number of pixels or voxels (in candidate intermediate image M3)whose response values are greater than the threshold may be determinedas C3. If the largest count or number among C1, C2, and C3 is C2, thenthe candidate intermediate image M2 may be designated as theintermediate image. The response values of the candidate intermediateimage M2 may be regarded as target response values. In some embodiments,for each candidate intermediate image, an average response value of theresponse values in the candidate intermediate image may be determined,and thus, two or more average response values may be determinedcorresponding to the two or more candidate intermediate images may bedetermined. A largest average response value among the two or moreaverage response values may be determined, and a candidate intermediateimage that has the largest average response value may be designated asan intermediate image corresponding to the initial image.

In 6225, the initial image and the intermediate image may be fused toobtain a fusion image. In 6230, the fusion image may be inputted into atrained processing model to generate a target image associated with theone or more target objects.

In some embodiments, the processing device 120 a (e.g., the segmentationmodule 4520) may perform operation 6225. In some embodiments, theprocessing device 120 a (e.g., the segmentation module 4520) may performoperation 6230.

In some embodiments, to generate the target image, the initial image andthe intermediate image may be inputted in a parallel mode into twodifferent input channels of the trained processing model. Each of thetwo input channels may transmit a corresponding input image to thetrained processing model.

In some embodiments, to generate the target image, the fusion image maybe determined by processing a first value of each of the plurality offirst pixels or voxels of the initial image and a second value of acorresponding second pixel or voxel of the intermediate image. Forexample, a value of a pixel or voxel of the fusion image may bedetermined based on a sum or product of the first value and the secondvalue. The fusion image may be inputted into the trained processingmodel to generate the target image.

According to the process 600, the application mechanism of the trainedprocessing model may be optimized. Specifically, the applicationmechanism may include obtaining the fusion image by fusing the initialimage and the intermediate image, and inputting the fusion image intothe trained processing model to generate the target image. Moreover, thedetermination of the intermediate image may be performed by processingthe initial image using one or more filters. Therefore, the segmentationof the target objects (e.g., blood vessels) can be facilitated after theobtaining of the initial image, thereby simplifying the input dataacquisition process, improving image processing efficiency, andimproving the user experience.

It should be noted that the above descriptions are merely provided forthe purposes of illustration, and not intended to limit the scope of thepresent disclosure. For persons having ordinary skills in the art,multiple variations and modifications may be made under the teachings ofthe present disclosure. However, those variations and modifications donot depart from the scope of the present disclosure. In someembodiments, one or more operations may be omitted and/or one or moreadditional operations may be added. For example, operation 6210 andoperation 6220 may be combined into a single operation. As anotherexample, one or more other optional operations (e.g., a storingoperation) may be added elsewhere in the process 600. In the storingoperation, the processing device 120 may store information and/or data(e.g., the initial image, the intermediate image, the trained processingmodel, etc.) associated with the image processing system 100 in astorage device (e.g., the storage device 130) disclosed elsewhere in thepresent disclosure.

FIG. 7A is a flowchart illustrating an exemplary process for determininga target image associated with one or more target objects according tosome embodiments of the present disclosure. The process 700 a mayinclude more descriptions of one or more operations in FIGS. 5-6.

In some embodiments, the process 700 a may be implemented as a set ofinstructions (e.g., an application) stored in the storage device 130,storage 220, or storage 390. The processing device 120, the processor210 and/or the CPU 340 may execute the set of instructions, and whenexecuting the instructions, the processing device 120, the processor 210and/or the CPU 340 may be configured to perform the process 700 a. Theoperations of the illustrated process presented below are intended to beillustrative. In some embodiments, the process 700 a may be accomplishedwith one or more additional operations not described and/or without oneor more of the operations discussed. Additionally, the order of theoperations of process 700 a illustrated in FIG. 7A and described belowis not intended to be limiting. In some embodiments, one or moreoperations (e.g., 7320-7330) in process 700 a may be performed by theprocessing device 120 a. In some embodiments, one or more operations(e.g., 7310) in process 700 a may be performed by the processing device120 b or an external processing device associated with the imageprocessing system 100. For example, operation 7310 in process 700 a maybe performed by a processing device of a vendor who provides and/ormaintains such a processing model (e.g., a manufacturer of the imagingdevice 110). As another example, operations 7320-7330 in process 700 amay be performed by a processing device of a client of the vendor.

In some embodiments, before operation 5120 “generating a target imageassociated with a target object using a trained processing model,” anadditional operation 7310 “training an initial processing model toobtain a trained processing model” may be added. In some embodiments,the operation 7310 “training an initial processing model to obtain atrained processing model” may include operations “determining aplurality of intermediate sample images based on a plurality of initialsample images; determining a predicted image by inputting each of theplurality of initial sample images and a corresponding intermediatesample image into the initial processing model; and adjusting orupdating one or more parameters of the initial processing model based ona difference between each of the plurality of predicted images and acorresponding reference image.” The operations may be performed toimprove the training effect of the initial processing model.

In some embodiments, as shown in FIG. 7A, an exemplary target imagegeneration process is provided. The process 700 a may include one ormore of the following operations.

In 7310, an initial processing model may be trained to obtain a trainedprocessing model.

In some embodiments, the processing device 120 a (e.g., the trainingmodule 4530) or the processing device 120 b may perform operation 7310.More descriptions of the training process of the initial processingmodel may be found elsewhere in the present disclosure (e.g., FIG. 7Band descriptions thereof).

In 7320, an initial image and/or an intermediate image corresponding tothe initial image may be obtained.

In some embodiments, the processing device 120 a (e.g., the obtainingmodule 4510) may perform operation 7320. Operation 7320 may be similarto operations 5110, 5115, 6210, 6220, and 8410-8455. More descriptionsabout the obtaining of the initial image and the intermediate image maybe found elsewhere in the present disclosure (e.g., FIGS. 5, 6, and 8and the descriptions thereof).

In 7330, a target image associated with one or more target objects maybe generated by inputting the initial image and the intermediate imageinto the trained processing model.

In some embodiments, the processing device 120 a (e.g., the segmentationmodule 4520) may perform operation 7330. Operation 7330 may be similarto operations 5120, 6225-6230, and 8460. More descriptions of thegeneration of the target image may be found elsewhere in the presentdisclosure (e.g., FIGS. 5, 6, and 8 and the descriptions thereof).

In some embodiments, operation 7310 may be performed before or afteroperation 7320. The order of operations 7310 and 7320 of the process 700a illustrated in FIG. 7A is not intended to be limiting.

In some embodiments, the initial image and the intermediate image may beused as training samples to update the trained processing model.

It should be noted that the above descriptions are merely provided forthe purposes of illustration, and not intended to limit the scope of thepresent disclosure. For persons having ordinary skills in the art,multiple variations and modifications may be made under the teachings ofthe present disclosure. However, those variations and modifications donot depart from the scope of the present disclosure. In someembodiments, one or more operations may be omitted and/or one or moreadditional operations may be added. For example, operation 7320 andoperation 7330 may be combined into a single operation. As anotherexample, one or more other optional operations (e.g., a storingoperation) may be added elsewhere in the process 700 a. In the storingoperation, the processing device 120 may store information and/or data(e.g., the initial image, the intermediate image, the trained processingmodel, etc.) associated with the image processing system 100 in astorage device (e.g., the storage device 130) disclosed elsewhere in thepresent disclosure.

FIG. 7B is a flowchart illustrating an exemplary process for training aprocessing model according to some embodiments of the presentdisclosure. In some embodiments, as shown in FIG. 7B, an exemplary modeltraining process is provided. The process 700 b may include one or moreof the following operations.

In some embodiments, the process 700 b may be implemented as a set ofinstructions (e.g., an application) stored in the storage device 130,storage 220, or storage 390. The processing device 120, the processor210 and/or the CPU 340 may execute the set of instructions, and whenexecuting the instructions, the processing device 120, the processor 210and/or the CPU 340 may be configured to perform the process 700 b. Theoperations of the illustrated process presented below are intended to beillustrative. In some embodiments, the process 700 b may be accomplishedwith one or more additional operations not described and/or without oneor more of the operations discussed. Additionally, the order of theoperations of process 700 b illustrated in FIG. 7B and described belowis not intended to be limiting. In some embodiments, the trainedprocessing model described elsewhere in the present disclosure (e.g.,operation 5120 in FIG. 5, operation 6230 in FIG. 6, operation 7330 inFIG. 7A, operation 8460 in FIG. 8) may be obtained according to theprocess 700 b. In some embodiments, the process 700 b may be performedby the processing device 120 b. In some embodiments, the process 700 bmay be performed by another processing device or a system other than theimage processing system 100, e.g., a processing device or system of avendor of a manufacturer. In the following descriptions, one or moreoperations of process 700 b performed by the processing device 120 b aremerely provided for the purposes of illustration, and not intended tolimit the scope of the present disclosure.

In 7305, an initial processing model may be obtained. In someembodiments, the processing device 120 b (e.g., the initial sample imageobtaining unit 410) may perform operation 7305.

In some embodiments, the initial processing model may include aconvolutional neural network (CNN) model (e.g., a U-Net neural networkmodel, or a V-Net neural network model), a deep CNN (DCNN) model, afully convolutional network (FCN) model, a recurrent neural network(RNN) model, or the like, or any combination thereof. In someembodiments, the initial processing model may be stored in one or morestorage devices (e.g., the storage device 130, the storage 220, and/orthe storage 390) associated with the image processing system 100 and/oran external data source. Accordingly, the initial processing model maybe retrieved from the storage devices and/or the external data source.

In some embodiments, the processing device 120 b may perform a pluralityof iterations to iteratively update one or more parameter values of theinitial processing model. Before the plurality of iterations, theprocessing device 120 b may initialize the parameter values of theinitial processing model. Exemplary parameters of the initial processingmodel may include the size of a kernel of a layer, the total count (ornumber) of layers, the count (or number) of nodes in each layer, alearning rate, a batch size, an epoch, a connected weight between twoconnected nodes, a bias vector relating to a node, etc. In someembodiments, the one or more parameters may be set randomly. In someembodiments, the one or more parameters may be set to one or morecertain values, e.g., 0, 1, or the like. In some embodiments, theparameter values of the initial processing model may be initializedbased on a Gaussian random algorithm, a Xavier algorithm, etc.

In 7311, a plurality of intermediate sample images may be determinedbased on a plurality of initial sample images. Each of the plurality ofintermediate sample images may correspond to one of the plurality ofinitial sample images.

In some embodiments, operation 7311 may be performed by the processingdevice 120 b (e.g., the intermediate sample image determination unit420). In some embodiments, the trained processing model may be obtainedby training the initial processing model in a training process based ona plurality of training samples. The initial sample images and thecorresponding intermediate sample images may be used as input of theinitial processing model in the training process of the initialprocessing model.

The plurality of training samples may include a plurality of initialsample images and a plurality of intermediate sample imagescorresponding to the plurality of initial sample images. The pluralityof training samples may associate with a sample organ or tissue. Thetraining samples may include one or more sample objects with a shapesimilar to or the same as the target objects. For example, if the targetobjects include blood vessels, the training samples may include sampleobjects with a linear shape. In some embodiments, an initial sampleimage may correspond to a subject including one or more target objects,(e.g., a subject as described elsewhere in the present disclosure (e.g.,FIG. 5 and the descriptions thereof)). The initial sample image mayinclude a representation of the subject, and the correspondingintermediate sample image may include a coarse representation of thetarget objects. For example, the initial sample image may include arepresentation of the heart, and the intermediate sample image mayinclude a coarse representation of one or more coronary arteries of theheart. In some embodiments, the plurality of initial sample images maycorrespond to a same or similar type of subject (e.g., the heart), andthe plurality of intermediate sample images may correspond to a same orsimilar type of target objects (e.g., the coronary arteries).

In some embodiments, the initial sample images may include CT images, MRimages, DSA images, PET images, SPECT images, DR images, or the like.

In some embodiments, the plurality of training samples used to train theinitial processing model may correspond to a same type of target object(e.g., a coronary artery), and a trained processing model correspondingto the type of target object may be determined. Therefore, the trainedprocessing model may be targetedly used to segment the same type oftarget object, thereby improving the segmentation accuracy of this typeof target object. For example, a coronary artery segmentation model maybe trained using a plurality of coronary artery sample images. Asanother example, a choroidal blood vessel segmentation model may betrained using a plurality of choroidal blood vessel sample images. As afurther example, a hepatic vein segmentation model may be trained usinga plurality of hepatic vein sample images.

In some embodiments, to determine the plurality of intermediate sampleimages, one or more filters may be used to process the plurality ofinitial sample images. In some embodiments, the one or more filters mayinclude a Gaussian filter, a tubular filter, a linear filter, a Wienerfilter, a threshold filter, or the like, or any combination thereof.

In some embodiments, to determine an intermediate sample image byprocessing an initial sample image based on the one or more filters, aHessian matrix corresponding to each pixel or voxel of the initialsample image may be determined, and at least one characteristic valuecorresponding to the each pixel or voxel may be determined based on theHessian matrix. A response value corresponding to the each pixel orvoxel may be determined by enhancing the at least one characteristicvalue corresponding to the each pixel or voxel using a first filter(e.g., a tubular filter or a linear filter) of the one or more filters.The intermediate sample image may be generated based on a plurality ofresponse values corresponding to a plurality of pixels or voxels of theinitial sample image.

In some embodiments, before determining the at least one characteristicvalue of the Hessian matrix corresponding to the each pixel or voxel,the initial sample image may be smoothed using a second filter (e.g., aGaussian filter) of the one or more filters to improve a precisionand/or accuracy of the intermediate sample image, thereby improving asegmentation accuracy of the trained processing model. The intermediatesample image may be generated based on the smoothed initial sampleimage. The second filter may include a Gaussian filter, a tubularfilter, a linear filter, a Wiener filter, a threshold filter, or thelike, or any combination thereof.

In some embodiments, in order to improve the precision and/or accuracyof the intermediate sample image, in the generation of the intermediatesample image based on the plurality of response values corresponding tothe plurality of pixels or voxels of the initial sample image, theinitial sample image may be smoothed using two or more second filters(e.g., Gaussian filters with kernels of different sizes) to generate twoor more smoothed initial sample images. Accordingly, two or morecandidate intermediate sample images may be generated based on the twoor more smoothed initial sample images, respectively. For each candidateintermediate sample image, an average response value of the responsevalues in the candidate intermediate sample image may be determined, andthus, two or more average response values may be determinedcorresponding to the two or more candidate intermediate sample imagesmay be determined. A largest average response value among the two ormore average response values may be determined, and a candidateintermediate sample image that has the largest average response valuemay be designated as an intermediate sample image corresponding to theinitial sample image.

Merely by way of example, an initial sample image may be smoothed bythree second filters, and accordingly, three smoothed initial sampleimages (e.g., SS1, SS2, SS3) may be generated. A candidate intermediatesample image may be generated based on each of the three smoothedinitial sample images, and accordingly, three candidate intermediatesample images (e.g., MS1 (corresponding to SS1), MS2 (corresponding toSS2), MS3 (corresponding to SS3)) may be generated. A first averageresponse value of the response values in the candidate intermediatesample image MS1 may be determined as AR1. A second average responsevalue of the response values in the candidate intermediate sample imageMS2 may be determined as AR2. A third average response value of theresponse values in the candidate intermediate sample image MS3 may bedetermined as AR3. If the largest average response value among AR1, AR2,and AR3 is AR1, then the candidate intermediate sample image MS1 may bedesignated as the intermediate sample image corresponding to the initialsample image.

It should be noted that the generation of the intermediate sample imagesbased on the initial sample images may be similar to or the same as thegeneration of the intermediate image based on the initial images asdescribed in connection with operation 6220 as illustrated in FIG. 6 (oraccording to one or more operations in process 800 as described in FIG.8).

In some embodiments, in order to improve an accuracy or precision thetrained processing model for generating a target image associated withthe one or more target objects, the generation of the intermediateimages may be the same as the generation of the intermediate sampleimages.

In 7312, a predicted image may be determined (e.g., in a currentiteration) by inputting one (e.g., each) of the plurality of initialsample images and a corresponding intermediate sample image into theinitial processing model.

In some embodiments, operation 7312 may be performed by the processingdevice 120 b (e.g., the predicted image determination unit 430).

Specifically, in some embodiments, at least one of the plurality ofinitial sample images and the corresponding intermediate sample imagemay be used as at least one training sample. A plurality of trainingsamples may be inputted into the initial processing model and aplurality of predicted images may be outputted from the initialprocessing model.

In some embodiments, each of the plurality of initial sample images andthe corresponding intermediate sample image may be fused to determine afusion sample image, and the fusion sample image may be inputted intothe initial processing model to train the initial processing model.

In some embodiments, each of the plurality of initial sample images andthe corresponding intermediate sample image may be inputted in aparallel mode into two different input channels of the initialprocessing model. Alternatively, or additionally, the fusion sampleimage may be determined based on a sum or product of a first pixel orvoxel value of each pixel or voxel in the initial sample image and asecond pixel or voxel value of each pixel or voxel in the correspondingintermediate sample image. The fusion sample image may be inputted intothe initial processing model to train the initial processing model.

In some embodiments, in order to improve an accuracy or precision of thetrained processing model for generating the target image associated withthe one or more target objects, the fusion operation of the initialimage and the intermediate image as described in FIG. 6 may be the sameas the fusion operation of the initial sample image and the intermediatesample image performed in the training process of the processing model.

Merely by way of example, if the initial sample image and theintermediate sample image are inputted in a parallel mode into twodifferent input channels of the initial processing model in the trainingprocess of the processing model, then in the generation of the targetimage (e.g., in FIG. 6), the initial image and the intermediate imagemay be inputted in a parallel mode into two different input channels ofthe trained processing model. As another example, if the initial sampleimage and the intermediate sample image are fused as a fusion sampleimage, and the fusion sample image is inputted into the initialprocessing model to train the initial processing model, then in thegeneration of the target image (e.g., in FIG. 6), the initial image andthe intermediate image may be fused as a fusion image and the fusionimage may be inputted into the trained processing model.

In 7313, one or more parameters of the initial processing model may beadjusted and/or updated based on a difference between each of theplurality of predicted images and a corresponding reference image.

In some embodiments, a trained processing model may be determined basedon the one or more updated parameters. In some embodiments, operation7313 may be performed by the processing device 120 b (e.g., theparameter adjustment unit 440).

In some embodiments, the corresponding reference image may refer to adesired target image corresponding to the each of the plurality ofinitial sample images. In some embodiments, the processing device 120 bmay obtain a plurality of reference images corresponding to the initialsample images from a storage device (e.g., the storage device 130, thestorage 220, the storage 390, or an external data source, or the like).In some embodiments, the reference images may be generated based oninitial sample images using one or more processing algorithms (e.g., asegmentation algorithm), and/or labelled or verified manually by a userof the image processing system 100. In some embodiments, a predictedimage corresponding to an initial sample image may be compared with areference image corresponding to the initial sample image, and adifference between the reference image and the predicted image may bedetermined. In some embodiments, an accuracy of the processing model inthe current iteration may be determined based on the difference. If theaccuracy satisfies a predetermined accuracy threshold, it may bedetermined that the processing model in the current iteration issufficiently trained, and the training process may be terminated. If theaccuracy does not satisfy the predetermined accuracy threshold, the oneor more parameters of the processing model in the current iteration maybe adjusted and/or updated based on the difference between the referenceimage and the predicted image to decrease the difference. In someembodiments, the parameters of the processing model may be adjustedand/or updated in a plurality of iterations until the accuracy satisfiesthe predetermined accuracy threshold.

In some embodiments, the reference images may be medical gold standardimages corresponding to the initial sample images. The predeterminedaccuracy threshold may be set according to a default setting of theimage processing system 100 or preset by a user or operator via theterminals(s) 140. In some embodiments, the predetermined accuracythreshold may be set according to an empirical value.

In some embodiments, the accuracy of the processing model in the currentiteration may be determined based on a cost function. The cost functionmay include a log loss function, a cross-entropy loss function, aleast-squares function, an index loss function, etc. In someembodiments, a plurality of iterations may be performed to update theone or more parameters of the processing model until a terminationcondition is satisfied. The termination condition may provide anindication of whether the initial processing model is sufficientlytrained. The termination condition may relate to the cost function or aniteration count of the training process. For example, the terminationcondition may be satisfied if the value of the cost function (e.g., theaccuracy) of the processing model is minimal or smaller than a threshold(e.g., a constant). As another example, the termination condition may besatisfied if the value of the cost function converges. The convergencemay be deemed to have occurred if the variation of the values of thecost function in two or more consecutive iterations is smaller than athreshold (e.g., a constant). As a further example, the terminationcondition may be satisfied when a specified number (or count) ofiterations are performed in the training process. As illustrated above,if the termination condition is not satisfied (e.g., if the accuracydoes not satisfy the predetermined accuracy threshold), in nextiteration(s), other training sample(s) may be inputted into theprocessing model to train the processing model as described above, untilthe termination condition is satisfied. The trained processing model maybe determined based on the one or more updated parameters. In someembodiments, the trained processing model may be transmitted to thestorage device 130, or any other storage device for storage.

In the present disclosure, to obtain the trained processing model, theplurality of intermediate sample images may be determined based on theplurality of initial sample images, and the plurality of initial sampleimages and the corresponding intermediate sample images may be inputtedinto the processing model to obtain a plurality of predicted imagescorresponding to the plurality of initial sample images. The one or moreparameters of the processing model may be adjusted and/or updated basedon the differences between the plurality of predicted images and thecorresponding reference images. According to the training processillustrated above, the training mechanism of the processing model may beclarified, and an effective segmentation of target objects in an initialimage may be realized using the trained processing model, therebyimproving the segmentation accuracy of the target objects (e.g., bloodvessels).

In some embodiments, the initial processing model may include twodifferent stages. If an image is input into a stage of a network, aninput channel may be formed. Each stage may correspond to a segmentationnetwork. For example, a first stage of the initial processing model maycorrespond to an initial coarse segmentation network, and a second stageof the initial processing model may correspond to an initial finesegmentation network. In some embodiments, as described in FIG. 5, thetrained processing model may include a coarse segmentation networkand/or a fine segmentation network. In some embodiments, the coarsesegmentation network and the fine segmentation network may be determinedby training two initial processing models in two training processes,respectively. For example, the initial coarse segmentation network maybe trained using initial sample images in a first training process. Asanother example, the initial fine segmentation network may be trainedusing intermediate sample images and extracted regions (or pixels orvoxels thereof) in the initial sample images corresponding to the targetobjects in the preliminary segmentation images in a second trainingprocess. The two training processes may be performed in a sameprocessing device or different processing devices. For example, thecoarse segmentation network may be determined in a first processingdevice (e.g., the processing device 120), while the fine segmentationnetwork may be determined in a second processing device (e.g., aprocessing device of a vendor of the processing model). As anotherexample, a specific processing device may first determine the coarsesegmentation network, and then determine the fine segmentation network.The trained processing model may be determined (or obtained, retrieved)based on the coarse segmentation network and the fine segmentationnetwork.

It should be noted that, the training processes for training the initialcoarse segmentation network and/or the initial fine segmentation networkmay be the same as or similar to the training process for training theinitial processing model. For example, an initial sample image may beinputted into the initial coarse segmentation network. A predictedpreliminary segmentation image may be output by the initial coarsesegmentation network. One or more parameters of the initial coarsesegmentation network may be adjusted and/or updated based on adifference between the predicted preliminary segmentation image and acorresponding desired preliminary segmentation image, until atermination condition is satisfied. The coarse segmentation network maybe determined based on updated parameters. As another example, anintermediate sample image and extracted region(s) (or pixels or voxelsthereof) in an initial sample image corresponding to the targetobject(s) in a corresponding preliminary segmentation image may beinputted into the initial fine segmentation network. A predicted imagemay be output by the initial fine segmentation network. One or moreparameters of the initial fine segmentation network may be adjustedand/or updated based on a difference between the predicted image and acorresponding desired target image, until a termination condition issatisfied. The fine segmentation network may be determined based onupdated parameters. In some embodiments, the termination conditions fortraining the coarse segmentation network and/or training the finesegmentation network may be similar to the termination condition fortraining the processing model.

In some alternative embodiments, the initial coarse segmentation networkand the initial fine segmentation network may be trained jointly in asingle training process. For example, the initial sample images may beinputted into the initial coarse segmentation network, and theintermediate sample images may be inputted into the initial finesegmentation network. The initial processing model (i.e., the initialcoarse segmentation network and the initial fine segmentation network)may process the input images (e.g., the initial sample images, theintermediate sample images). In some embodiments, the initial processingmodel may extract one or more image features (e.g., a low-level feature(e.g., an edge feature, a texture feature), a high-level feature (e.g.,a semantic feature), or a complicated feature (e.g., a deep hierarchicalfeature) included in the inputted images. Based on the extracted imagefeatures, the initial processing model may determine the predictedimages corresponding to the inputted images. The parameters of theinitial processing model may be adjusted and/or updated based ondifferences between the predicted images and corresponding referenceimages, until a termination condition is satisfied. The trainedprocessing model (i.e., the coarse segmentation network and the finesegmentation network) may be determined based on the updated parameters.

The trained processing model may include two different stages. The firststage of the trained processing model may correspond to the coarsesegmentation network, and the second stage of the trained processingmodel may correspond to the fine segmentation network. In someembodiments, the coarse segmentation network may include one inputchannel, initial images may be inputted into the input channel, andpreliminary segmentation images may be outputted from the output channelof the coarse segmentation network. In some embodiments, the finesegmentation network may include two input channels, the extractedregion(s) (or pixels or voxels thereof) in the initial imagescorresponding to the target object(s) in the preliminary segmentationimages and the intermediate images may be inputted into the two inputchannels, respectively, and target images may be outputted from the finesegmentation network. In some embodiments, the output of the coarsesegmentation network may be input in the fine segmentation network.

It should be noted that the above descriptions are merely provided forthe purposes of illustration, and not intended to limit the scope of thepresent disclosure. For persons having ordinary skills in the art,multiple variations and modifications may be made under the teachings ofthe present disclosure. However, those variations and modifications donot depart from the scope of the present disclosure. In someembodiments, one or more operations may be omitted and/or one or moreadditional operations may be added. For example, the operation 7312 andthe operation 7313 may be combined into a single operation to update theparameters of the processing model. As another example, one or moreother optional operations (e.g., a storing operation) may be addedelsewhere in the process 700 b. In the storing operation, the processingdevice 120 b may store information and/or data (e.g., the initial sampleimages, the intermediate sample images, the desired target images, theparameters, etc.) associated with the image processing system 100 in astorage device (e.g., the storage device 130) disclosed elsewhere in thepresent disclosure.

FIG. 8 is a flowchart illustrating an exemplary process for determininga target image associated with one or more target objects according tosome embodiments of the present disclosure. In some embodiments, forillustration purpose, a process for segmenting a plurality of bloodvessels (e.g., a coronary artery) from a heart CT image is taken as anexample in the following descriptions.

In some embodiments, the process 800 may be implemented as a set ofinstructions (e.g., an application) stored in the storage device 130,storage 220, or storage 390. The processing device 120, the processor210 and/or the CPU 340 may execute the set of instructions, and whenexecuting the instructions, the processing device 120, the processor 210and/or the CPU 340 may be configured to perform the process 800. Theoperations of the illustrated process presented below are intended to beillustrative. In some embodiments, the process 800 may be accomplishedwith one or more additional operations not described and/or without oneor more of the operations discussed. Additionally, the order of theoperations of process 800 illustrated in FIG. 8 and described below isnot intended to be limiting. In some embodiments, the intermediate image(or intermediate sample image) described elsewhere in the presentdisclosure (e.g., operation 5115 illustrated in FIG. 5, operation 6220illustrated in FIG. 6, and/or operation 7311 illustrated in FIG. 7B) maybe obtained according to one or more operations (e.g., 8420-8455) in theprocess 800.

In some embodiments, as shown in FIG. 8, an exemplary blood vessel imagesegmentation process is provided. The process 800 may include one ormore of the following operations.

In 8410, an initial image may be obtained. In some embodiments, theinitial image may be a heart CT image as shown in FIG. 11A.

In some embodiments, operation 8410 may be performed by the processingdevice 120 a (e.g., the obtaining module 4510). The initial image mayinclude representations of one or more target objects (e.g., a coronaryartery) and a plurality of interfering objects. More descriptions of theinitial image and the obtaining of the initial image may be foundelsewhere in the present disclosure (e.g., FIG. 5 and descriptionsthereof).

In 8420, one or more smoothed initial images may be generated bysmoothing the initial image using one or more second filters.

In some embodiments, operation 8420 may be performed by the processingdevice 120 a (e.g., the obtaining module 4510). In some embodiments, theone or more second filters may include a Gaussian, a linear filter, aWiener filter, a threshold filter, or the like, or any combinationthereof.

In some embodiments, the one or more second filters may include filterswith kernels of different sizes. For example, the one or more secondfilters may include Gaussian filters with kernels of different sizes,such as 2×2, 3×3, 4×4, 5×5, 6×6, 7×7, 8×8, 9×9, etc. In someembodiments, the initial image may be smoothed using the one or moreGaussian filters to generate the one or more smoothed initial images. Insome embodiments, by smoothing the initial image using the one or moreGaussian filters, a sensitivity of data associated with the initialimage (e.g., data of Hessian matrixes described in operation 8430, dataof an intermediate image described in operation 8460, etc.) to Gaussiannoises may be reduced or eliminated in the subsequent processingoperations. The one or more Gaussian filters with kernels of differentsizes may be adapted to different sizes of the one or more targetobjects. In some embodiments, if the initial image includes arepresentation of a blood vessel, the one or more target objects mayrefer to different portions of the blood vessel to be segmented from theinitial image.

As used herein, the one or more Gaussian filters are adapted todifferent sizes of the one or more target objects may refer that aspecific filter of the one or more Gaussian filters may have a goodsensitivity and/or a good specificity in processing a specific targetobject of a specific size in the one or more target objects. In someembodiments, if the initial image is a 3D image, the initial image maybe smoothed (e.g., sequentially) using three Gaussian filters withkernels of different sizes (e.g., 3×3, 5×5, 7×7) in three differentdirections (e.g., an X-axis direction, a Y-axis direction, and a Z-axisdirection) of the initial image, and a corresponding 3D smoothed initialimage may be generated.

In 8430, a Hessian matrix corresponding to each pixel or voxel of eachsmoothed initial image of the one or more smoothed initial images may bedetermined. In 8435, one or more characteristic values corresponding tothe each pixel or voxel of the each smoothed initial image may bedetermined (e.g., using the Jacobi algorithm) based on the Hessianmatrix.

In some embodiments, operation 8430 may be performed by the processingdevice 120 a (e.g., the obtaining module 4510). In some embodiments,operation 8435 may be performed by the processing device 120 a (e.g.,the obtaining module 4510). In some embodiments, the at least onecharacteristic value corresponding to the each pixel or voxel of theeach smoothed initial image may be determined based on one or morealgorithms. For example, the at least one characteristic value may bedetermined using the Jacobi algorithm. As another example, the at leastone characteristic value may be determined using an iterative techniqueof diagonalizing the Hessian matrix. As a further example, the at leastone characteristic value may be analytically determined by directlysolving the relationship Hu=λu, in which H is the Hessian matrix, u isan eigenvector of the Hessian matrix, and λ is an characteristic valueassociated with u.

In some embodiments, if a smoothed initial image is a 3D image, eachvoxel of the 3D smoothed initial image may correspond to threecharacteristic values.

In some embodiments, if a smoothed initial image is a 2D image, eachpixel of the 2D smoothed initial image may correspond to twocharacteristic values.

In 8440, a response value corresponding to the each pixel or voxel ofthe each smoothed initial image may be determined by enhancing the oneor more characteristic values corresponding to the each pixel or voxelof the each smoothed initial image using a first filter.

In some embodiments, operation 8440 may be performed by the processingdevice 120 a (e.g., the obtaining module 4510). In some embodiments, thefirst filter may be selected based on shape characteristics of the oneor more target objects. For example, if the one or more target objectshave a linear shape, the first filter may include a tubular filter, alinear filter, etc. If the one or more target objects have a dot shape,the first filter may include a dot filter. In some embodiments, atubular filter (or a linear filter) may be selected as the first filterfor blood vessel segmentation.

Using the tubular filter, a specific response value corresponding to aspecific pixel or voxel of a smoothed initial image located inside aregion of the target objects (e.g., blood vessels) may be greater thananother specific response value corresponding to another specific pixelor voxel of the smoothed initial image that is located outside theregion of the target objects. In some embodiments, at least a portion ofthe interfering objects represented in the smoothed initial images (orthe initial images) may be deleted or removed by adjusting or setting asuitable bandwidth of the tubular filter.

In some embodiments, the bandwidth of the tubular filter may be setbased on a size range of the one or more target objects. For example, ifa minimum size of a target object is 1 millimeter and a maximum size ofthe target object is 6 millimeters, then the bandwidth of the tubularfilter may be set as 1 millimeter to 6 millimeters. In some embodiments,the bandwidth of the tubular filter may be set according to a defaultsetting of the image processing system 100 or preset by a user oroperator via the terminals(s) 140.

In some embodiments, for a 3D smoothed initial image, a response valuecorresponding to a pixel or voxel of the 3D smoothed initial image maybe determined according to Equation (1) as follows:

$\begin{matrix}{{F = {\frac{{a_{2}} \times \left( {{a_{2}} - {a_{3}}} \right)}{a_{1}} \times \sigma^{2}}},} & (1)\end{matrix}$

where a₁, a₂, and a₃ refer to the three characteristic valuescorresponding to the pixel or voxel, respectively; F refers to theresponse value corresponding to the pixel or voxel; and a refers to thebandwidth of the first filter.

In 8450, a maximum response value corresponding to each pixel or voxelof the initial image may be selected among one or more response valuescorresponding to pixels or voxels at a same position in the one or moresmoothed initial images. In 8455, an intermediate image may be generatedbased on a plurality of maximum response values corresponding to aplurality of pixels or voxels of the initial image.

In some embodiments, operation 8450 may be performed by the processingdevice 120 a (e.g., the obtaining module 4510). In some embodiments,operation 8455 may be performed by the processing device 120 a (e.g.,the obtaining module 4510).

An exemplary intermediate image may be shown in FIG. 11B. Becauseresponse values corresponding to pixels or voxels of at least a portionof the interfering objects in the intermediate image are smaller thanresponse values corresponding to pixels or voxels of the target objects,pixels or voxels of the at least a portion of the interfering objectsmay have relatively low brightness, while pixels or voxels of the one ormore target objects may have relatively high brightness. In someembodiments, due to a limitation of the first filter (e.g., the tubularfilter), the intermediate image may also include representations of aportion of the interfering objects. For example, as shown in FIG. 11B,the intermediate image may have representations of one or more bones, atrachea, one or more blood vessels other than the coronary artery, etc.

In 8460, the initial image and the intermediate image may be used asinput images and may be inputted into a trained processing model togenerate a target image associated with the one or more target objects.The trained processing model may be obtained by training an initialprocessing model in a training process. The processing model may includea convolutional neural network (CNN) model (e.g., a U-Net neural networkmodel, or a V-Net neural network model), a deep CNN (DCNN) model, afully convolutional network (FCN) model, a recurrent neural network(RNN) model, or the like, or any combination thereof.

More descriptions of the application of the trained processing mode maybe found elsewhere in the present disclosure (e.g., FIG. 9 anddescriptions thereof). The processing model may include at least twoinput channels. A first input channel may be configured to input theinitial (sample) image(s), and a second input channel may be configuredto input the intermediate (sample) image(s). In the training process ofthe processing model, the processing model may learn rich features fromthe initial sample images and the intermediate sample images, therebyreducing learning difficulty, and improving learning efficiency. Inaddition, because a plurality of interfering objects are substantiallyremoved from the intermediate sample images, effective prior knowledge(or information) can be introduced in the training process of theprocessing model, thereby improving an accuracy of the trainedprocessing model.

In some embodiments, operation 8460 may be performed by the processingdevice 120 a (e.g., the segmentation module 4520). In some embodiments,the trained processing model may accordingly include a coarsesegmentation network and a fine segmentation network. In someembodiments, the initial image and the intermediate image may beinputted in a parallel mode into two different input channels of thetrained processing model to generate the target image. For example, theinitial image may be inputted into to the coarse segmentation network,and the intermediate image may be inputted into the fine segmentationnetwork. In some embodiments, the processing device 120 may fuse theinitial image and the intermediate image to obtain a fusion image. Theprocessing device 120 may input the fusion image into the trainedprocessing model to generate the target image. More descriptions fordetermining the target image based on the fusion image may be foundelsewhere in the present disclosure (e.g., FIG. 6 and the descriptionsthereof).

In some embodiments, a largest connected domain in a preliminarysegmentation image generated from the coarse segmentation network may beextracted as an updated preliminary segmentation image. In someembodiments, one or more updated extracted region(s) (with the originalresolution) in the initial image corresponding to the target object(s)in the updated preliminary segmentation image may be determined. Theupdated extracted region(s) (or pixels or voxels thereof) and theintermediate image may be inputted into the fine segmentation network togenerate the target image.

For illustration purposes, an exemplary target image generated accordingto the operations illustrated above may be shown in FIG. 12A. Anexemplary segmentation result generated based on a heart image ratherthan an intermediate image may be shown in FIG. 12A. An exemplarymedical gold standard image may be shown in FIG. 12A.

In 8470, a largest connected domain in the target image may be extractedas an updated target image.

In some embodiments, operation 8470 may be performed by the processingdevice 120 a (e.g., the segmentation module 4520).

In some embodiments, the target image generated in 8460 may include aplurality of image noises, and/or the representations of the targetobjects in the target image may include one or more broken branches(e.g., a coronary artery segmentation result as shown in image A in FIG.12B), and accordingly, the target image may have a relatively poorvisual effect. In some embodiments, to improve the visual effect of thetarget image, the largest connected domain in the target image may beextracted as the updated target image (e.g., image B in FIG. 12B), inwhich the image noises and the broken branches may be filtered orremoved.

It should be noted that the above descriptions are merely provided forthe purposes of illustration, and not intended to limit the scope of thepresent disclosure. For persons having ordinary skills in the art,multiple variations and modifications may be made under the teachings ofthe present disclosure. However, those variations and modifications donot depart from the scope of the present disclosure. In someembodiments, one or more operations may be omitted and/or one or moreadditional operations may be added. For example, operation 8440 andoperation 8450 may be combined into a single operation. As anotherexample, one or more other optional operations (e.g., a storingoperation) may be added elsewhere in the process 800. In the storingoperation, the processing device 120 may store information and/or data(e.g., the initial images, the intermediate images, the trainedprocessing model, etc.) associated with the image processing system 100in a storage device (e.g., the storage device 130) disclosed elsewherein the present disclosure.

FIG. 9 is a diagram illustrating an exemplary process for generating anintermediate image and a target image according to some embodiments ofthe present disclosure.

As illustrated in FIG. 9, an initial image may be first processed usingthree second filters (e.g., Gaussian filters with kernels of sizes 3×3,5×5, and 7×7), respectively, to generate three smoothed initial images.At least one characteristic value of a Hessian matrix corresponding toeach pixel or voxel of each of the three smoothed initial images may bedetermined. A response value corresponding to the each pixel or voxel ofeach of the three smoothed initial images may be determined by enhancingthe at least one characteristic value corresponding to the each pixel orvoxel of each of the three smoothed initial images using a first filter(e.g., a tubular filter or a linear filter). A maximum response valuecorresponding to each pixel or voxel of the initial image may beselected among three response values corresponding to pixels or voxelsat a same position in the three smoothed initial images, and anintermediate image may be generated based on a plurality of maximumresponse values corresponding to a plurality of pixels or voxels of theinitial image. The initial image and the intermediate image may beinputted into a trained processing model to obtain a target image (or acandidate target image). In some embodiments, a largest connected domainin the target image (or the candidate target image) may be extracted asan updated target image (or a target image).

FIG. 10 is a block diagram illustrating an exemplary medical deviceaccording to some embodiments of the present disclosure. As illustratedin FIG. 10, the medical device 1000 may include an input device 10610,an output device 10620, one or more processors 10630, and a storagedevice 10640.

The input device 10610 may be configured to obtain an initial image.

The output device 10620 may be configured to display the initial imageand/or a segmentation result (i.e., a target image) of one or moretarget objects.

The processor(s) 10630 may be configured to process data and/orinformation associated with the image processing system 100.

The storage device 10640 may be configured to store one or moreinstructions and/or programs.

As shown in FIG. 6, only one processor is illustrated as an example. Theinput device 10610 may be connected to the output device 10620, theprocessor 10630, and the storage device 10640 via a bus or any otherconnection manner. The processor 10630 and the storage device 10640 maybe connected via a bus or any other connection manner. For illustrationpurpose, the input device 10610, the output device 10620, the processor10630, and/or the storage device 10640 are connected with each other viaa bus.

In some embodiments, the processor 10630 may obtain the initial imageand an intermediate image corresponding to the initial image from theinput device 10610 and/or the storage device 10640. The processor 10630may input the initial image and/or the intermediate image into a trainedprocessing model stored in the storage device 10640 to generate thetarget image associated with the one or more target objects.

The storage device 10640 may be used as a computer readable storagemedium for storing one or more programs. The one or more programs may bea software program, a computer executable program and/or module, such asinstructions/modules (e.g., the obtaining module 4510 and/or thesegmentation module 4520 shown in FIG. 4A) corresponding to theprocess(es) in the present disclosure. The processor 10630 may implementvarious functions and data processing of the medical device by executingsoftware programs, instructions, and/or modules stored in the storagedevice 10640.

The storage device 10640 may include a program storage area and a datastorage area. The program storage area may store an operation system,one or more programs each of which corresponds to a specific function.The data storage area may store data such as the initial images, theintermediate images, the trained processing model, the target image,etc. In addition, the storage device 10640 may include a high speedrandom access memory, a non-volatile memory, (e.g., at least onemagnetic disk, a flash drive, or other non-volatile solid state storagedevice), etc. In some embodiments, the storage device 10640 may furtherinclude a memory remotely located relative to the processor 10630, whichmay be connected to a server via a network (e.g., the network 150). Thenetwork may include, but are not limited to, an Internet, an intranet, alocal area network (LAN), a mobile communication network, or the like,or any combination thereof.

It should be noted that the above descriptions are merely provided forthe purposes of illustration, and not intended to limit the scope of thepresent disclosure. For persons having ordinary skills in the art,multiple variations and modifications may be made under the teachings ofthe present disclosure. However, those variations and modifications donot depart from the scope of the present disclosure.

EXAMPLES

The following examples are provided for illustration purposes and arenot intended to limit the scope of the present disclosure.

Example 1 Exemplary Heart Image and Coarse Blood Vessel ImageCorresponding to the Heart Image

FIG. 11A shows an exemplary initial image according to some embodimentsof the present disclosure. FIG. 11B shows an exemplary intermediateimage corresponding to the initial image according to some embodimentsof the present disclosure. The intermediate image may be determined byprocessing the initial image using one or more filters described in thepresent disclosure. As shown in FIG. 11A, the initial image is an imageof a heart (also referred to as a heart image). The heart image (i.e.,the initial image) includes representations of the heart, the spleen,the stomach, bones, a plurality of blood vessels, etc. As shown in FIG.11B, the intermediate image mainly includes representations of a portionof the plurality of blood vessels (e.g., the coronary artery), and alsoincludes representations of a portion of interfering objects (e.g., oneor more bones, tracheas, etc.). One or more main interfering objects(i.e., non-blood vessel objects, such as the heart, the spleen, thestomach, etc.) have low response values in the intermediate image, i.e.,values of pixels corresponding to the main interfering objects are lowerthan values of pixels corresponding to the blood vessels. That is, theintermediate image illustrates a coarse representation of a portion ofthe blood vessels.

Example 2 Exemplary Coronary Artery Images Generated Based on a SingleSource Model and a Multi-Source Model

FIG. 12A shows an exemplary image generated based on a single sourcemodel, an exemplary image generated based on a multi-source model, andan exemplary gold standard image according to some embodiments of thepresent disclosure. As illustrated in FIG. 12A, image 1 is a coronaryartery image generated by processing a heart image using a single sourcemodel (i.e., only the heart image is inputted into the single sourcemodel (i.e., a segmentation network (e.g., a coarse segmentationnetwork, a fine segmentation network)) to generate the image 1). Image 2is a coronary artery image generated by processing the heart image usinga multi-source model (i.e., the trained processing model described inthe present disclosure, and the heart image and a coarse blood vesselimage corresponding to the heart image are inputted into the trainedprocessing model to generate the image 2). Image 3 is a gold standardcoronary artery image corresponding to the heart image. In comparisonwith image 3, image 1 (as marked by the dotted frame 1210 in image 1)misses a portion of the coronary artery (as marked by the dotted frame1230 in image 3), and image 2 includes the corresponding portion (asmarked by the dotted frame 1220 in image 2). That is, the multi-sourcemodel (i.e., the trained processing model described in the presentdisclosure) can improve the segmentation accuracy of the coronaryartery.

Example 3 Exemplary Coronary Artery Images Generated with and withoutExtracting a Largest Connected Domain

FIG. 12B shows an exemplary image generated without extracting a largestconnected domain, an exemplary image generated with extracting thelargest connected domain, and an exemplary gold standard image accordingto some embodiments of the present disclosure. As illustrated in FIG.12B, image A is a coronary artery image generated by processing a heartimage using a trained processing model without extracting a largestconnected domain. Image B is a coronary artery image generated byprocessing the heart image using a single source model (i.e., only theheart image is inputted into the single source model (i.e., asegmentation network (e.g., a coarse segmentation network, a finesegmentation network)) to generate the image B) and extracting thelargest connected domain from the segmentation result. Image C is a goldstandard coronary artery image corresponding to the heart image. Asshown in FIG. 12B, image A includes representations of coronary arterieswith one or more broken branches and image noises (as marked by dottedframes 1240, 1250, and 1260). In comparison with image A, image B hasless noises and is similar to image C. That is, by extracting a largestconnected domain in the coronary artery image (i.e., the segmentationresult generated based on the trained processing model), the precisionand accuracy of the coronary artery image can be improved.

Having thus described the basic concepts, it may be rather apparent tothose skilled in the art after reading this detailed disclosure that theforegoing detailed disclosure is intended to be presented by way ofexample only and is not limiting. Various alterations, improvements, andmodifications may occur and are intended to those skilled in the art,though not expressly stated herein. These alterations, improvements, andmodifications are intended to be suggested by this disclosure and arewithin the spirit and scope of the exemplary embodiments of thisdisclosure.

Moreover, certain terminology has been used to describe embodiments ofthe present disclosure. For example, the terms “one embodiment,” “anembodiment,” and/or “some embodiments” mean that a particular feature,structure or characteristic described in connection with the embodimentis included in at least one embodiment of the present disclosure.Therefore, it is emphasized and should be appreciated that two or morereferences to “an embodiment” or “one embodiment” or “an alternativeembodiment” in various portions of this specification are notnecessarily all referring to the same embodiment. Furthermore, theparticular features, structures or characteristics may be combined assuitable in one or more embodiments of the present disclosure.

Further, it will be appreciated by one skilled in the art, aspects ofthe present disclosure may be illustrated and described herein in any ofa number of patentable classes or context including any new and usefulprocess, machine, manufacture, or composition of matter, or any new anduseful improvement thereof. Accordingly, aspects of the presentdisclosure may be implemented entirely hardware, entirely software(including firmware, resident software, micro-code, etc.) or combiningsoftware and hardware implementation that may all generally be referredto herein as a “unit,” “module,” or “system.” Furthermore, aspects ofthe present disclosure may take the form of a computer program productembodied in one or more computer-readable media having computer-readableprogram code embodied thereon.

A non-transitory computer-readable signal medium may include apropagated data signal with computer readable program code embodiedtherein, for example, in baseband or as part of a carrier wave. Such apropagated signal may take any of a variety of forms, includingelectromagnetic, optical, or the like, or any suitable combinationthereof. A computer-readable signal medium may be any computer-readablemedium that is not a computer-readable storage medium and that maycommunicate, propagate, or transport a program for use by or inconnection with an instruction execution system, apparatus, or device.Program code embodied on a computer-readable signal medium may betransmitted using any appropriate medium, including wireless, wireline,optical fiber cable, RF, or the like, or any suitable combination of theforegoing.

Computer program code for carrying out operations for aspects of thepresent disclosure may be written in any combination of one or moreprogramming languages, including an object-oriented programming languagesuch as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C #, VB.NET, Python or the like, conventional procedural programming languages,such as the “C” programming language, Visual Basic, Fortran, Perl,COBOL, PHP, ABAP, dynamic programming languages such as Python, Ruby,and Groovy, or other programming languages. The program code may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider) or in a cloud computing environment or offered as aservice such as a Software as a Service (SaaS).

Furthermore, the recited order of processing elements or sequences, orthe use of numbers, letters, or other designations, therefore, is notintended to limit the claimed processes and methods to any order exceptas may be specified in the claims. Although the above disclosurediscusses through various examples what is currently considered to be avariety of useful embodiments of the disclosure, it is to be understoodthat such detail is solely for that purpose and that the appended claimsare not limited to the disclosed embodiments, but, on the contrary, areintended to cover modifications and equivalent arrangements that arewithin the spirit and scope of the disclosed embodiments. For example,although the implementation of various components described above may beembodied in a hardware device, it may also be implemented as asoftware-only solution, e.g., an installation on an existing server ormobile device.

Similarly, it should be appreciated that in the foregoing description ofembodiments of the present disclosure, various features are sometimesgrouped together in a single embodiment, figure, or description thereofto streamline the disclosure aiding in the understanding of one or moreof the various inventive embodiments. This method of disclosure,however, is not to be interpreted as reflecting an intention that theclaimed object matter requires more features than are expressly recitedin each claim. Rather, inventive embodiments lie in less than allfeatures of a single foregoing disclosed embodiment.

In some embodiments, the numbers expressing quantities, properties, andso forth, used to describe and claim certain embodiments of theapplication are to be understood as being modified in some instances bythe term “about,” “approximate,” or “substantially.” For example,“about,” “approximate” or “substantially” may indicate ±20% variation ofthe value it describes, unless otherwise stated. Accordingly, in someembodiments, the numerical parameters set forth in the writtendescription and attached claims are approximations that may varydepending upon the desired properties sought to be obtained by aparticular embodiment. In some embodiments, the numerical parametersshould be construed in light of the number of reported significantdigits and by applying ordinary rounding techniques. Notwithstandingthat the numerical ranges and parameters setting forth the broad scopeof some embodiments of the application are approximations, the numericalvalues set forth in the specific examples are reported as precisely aspracticable.

Each of the patents, patent applications, publications of patentapplications, and other material, such as articles, books,specifications, publications, documents, things, and/or the like,referenced herein is hereby incorporated herein by this reference in itsentirety for all purposes, excepting any prosecution file historyassociated with same, any of same that is inconsistent with or inconflict with the present document, or any of same that may have alimiting effect as to the broadest scope of the claims now or laterassociated with the present document. By way of example, should there beany inconsistency or conflict between the description, definition,and/or the use of a term associated with any of the incorporatedmaterial and that associated with the present document, the description,definition, and/or the use of the term in the present document shallprevail.

In closing, it is to be understood that the embodiments of theapplication disclosed herein are illustrative of the principles of theembodiments of the application. Other modifications that may be employedmay be within the scope of the application. Thus, by way of example, butnot of limitation, alternative configurations of the embodiments of theapplication may be utilized in accordance with the teachings herein.Accordingly, embodiments of the present application are not limited tothat precisely as shown and described.

We claim:
 1. A method for image processing, comprising: obtaining aninitial image; obtaining an intermediate image corresponding to theinitial image, the intermediate image including pixels or voxelsassociated with at least a portion of a target object in the initialimage; obtaining a trained processing model; and generating, based onthe initial image and the intermediate image, a target image associatedwith the target object using the trained processing model.
 2. The methodof claim 1, wherein the obtaining an intermediate image comprises:generating the intermediate image by processing the initial image usingat least one filter.
 3. The method of claim 2, wherein the generatingthe intermediate image by processing the initial image using at leastone filter comprises: determining a Hessian matrix corresponding to eachpixel or voxel of the initial image; determining, based on the Hessianmatrix, at least one characteristic value corresponding to the eachpixel or voxel; determining a response value corresponding to the eachpixel or voxel by enhancing, using a first filter of the at least onefilter, the at least one characteristic value corresponding to the eachpixel or voxel; and generating, based on a plurality of response valuescorresponding to a plurality of pixels or voxels of the initial image,the intermediate image.
 4. The method of claim 3, wherein the firstfilter includes at least one of a Gaussian filter, a tubular filter, alinear filter, or a Wiener filter.
 5. The method of claim 3, wherein thegenerating the intermediate image by processing the initial image usingat least one filter further comprises: smoothing the initial image usinga second filter of the at least one filter.
 6. The method of claim 5,wherein the second filter includes at least one of a Gaussian filter, alinear filter, or a Wiener filter.
 7. The method of claim 2, wherein thegenerating the intermediate image by processing the initial image usingat least one filter comprises: generating one or more smoothed initialimages by smoothing the initial image using one or more second filtersof the at least one filter; determining a Hessian matrix correspondingto each pixel or voxel of each smoothed initial image of the one or moresmoothed initial images; determining, based on the Hessian matrix, atleast one characteristic value corresponding to the each pixel or voxelof the each smoothed initial image; determining a response valuecorresponding to the each pixel or voxel of the each smoothed initialimage by enhancing, using a first filter of the at least one filter, theat least one characteristic value corresponding to the each pixel orvoxel of the each smoothed initial image; determining, based on one ormore response values corresponding to pixels or voxels in the one ormore smoothed initial images, a target response value corresponding toeach pixel or voxel of the initial image; and generating, based on aplurality of target response values corresponding to a plurality ofpixels or voxels of the initial image, the intermediate image.
 8. Themethod of claim 1, wherein the generating a target image comprises:fusing the initial image and the intermediate image to obtain a fusionimage; and inputting the fusion image into the trained processing modelto generate the target image.
 9. The method of claim 8, wherein thefusing the initial image and the intermediate image comprises:generating the fusion image by processing a first value of each pixel orvoxel of the initial image with a second value of a corresponding pixelor voxel of the intermediate image.
 10. The method of claim 9, whereinthe processing a first value of each pixel or voxel of the initial imagewith a second value of a corresponding pixel or voxel of theintermediate image comprises: determining a value of a pixel or voxel ofthe fusion image based on a sum or product of the first value and thesecond value.
 11. The method of claim 1, wherein the generating a targetimage comprises: inputting, in a parallel mode, the initial image andthe intermediate image into two different input channels of the trainedprocessing model to generate the target image.
 12. The method of claim1, further comprising: obtaining an initial processing model; andtraining the initial processing model to obtain the trained processingmodel.
 13. The method of claim 1, wherein the trained processing modelis generated according to a process, the process including: obtaining aninitial processing model; obtaining a plurality of training samples, theplurality of training samples including a plurality of initial sampleimages and a plurality of intermediate sample images corresponding tothe plurality of initial sample images; and generating the trainedprocessing model by training the initial processing model using theplurality of training samples.
 14. The method of claim 1, wherein thetrained processing model is configured to segment, based on theintermediate image, the target object from the initial image.
 15. Themethod of claim 14, wherein the trained processing model includes acoarse segmentation network and a fine segmentation network.
 16. Themethod of claim 1, wherein the trained processing model is a trainedV-Net neural network model.
 17. The method of claim 1, furthercomprising: updating the target image, including: extracting a largestconnected domain in the target image as an updated target image.
 18. Themethod of claim 1, wherein the target object includes a blood vessel.19. A system for image processing, comprising: at least one storagedevice storing executable instructions, and at least one processor incommunication with the at least one storage device, when executing theexecutable instructions, causing the system to perform operationsincluding: obtaining an initial image; obtaining an intermediate imagecorresponding to the initial image, the intermediate image includingpixels or voxels associated with at least a portion of a target objectin the initial image; obtaining a trained processing model; andgenerating, based on the initial image and the intermediate image, atarget image associated with the target object using the trainedprocessing model.
 20. A non-transitory computer readable medium,comprising at least one set of instructions for image processing,wherein when executed by one or more processors of a computing device,the at least one set of instructions causes the computing device toperform a method, the method comprising: obtaining an initial image;obtaining an intermediate image corresponding to the initial image, theintermediate image including pixels or voxels associated with at least aportion of a target object in the initial image; obtaining a trainedprocessing model; and generating, based on the initial image and theintermediate image, a target image associated with the target objectusing the trained processing model.